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By 
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Major  Department:  Clinical  and  Health  Psychology 

The  primary  purposes  of  this  study  were  to  (1)  determine  if  controls,  mild,  and 

moderate/severe  traumatic  brain  injury  (TBI)  participants  performed  differently  on  a 

battery  of  executive  functioning  tests;  (2)  identify  the  operating  characteristics  of 

executive  functioning  tests  in  controls  and  traumatic  brain  injury  participants;  and  (3) 

develop  a  linear  combination  of  predictor  variables  that  distinguished  group  membership. 

Participants  consisted  of  46  brain  injured  individuals  and  24  non  brain  injured  controls. 

All  participants  completed  an  extensive  battery  of  executive  functioning  tests.  Multiple 

one-way  ANOVAs  showed  that,  in  general,  no  differences  in  test  performance  were  noted 

between  controls  and  mild  TBI  participants,  but  moderate/severe  TBI  participants 

consistently  performed  worse  than  either  group.  Results  also  showed  that  a  computerized 

version  of  the  Wisconsin  Card  Sorting  Test  yielded  the  highest  positive  predictive  power 

among  all  tests,  and  the  proportion  of  perseverative  errors  best  distinguished  groups  in  a 


discriminant  function  analysis.  These  findings  provide  information  regarding  the 
diagnostic  accuracy  of  executive  functioning  tests  in  TBI  and  should  assist  clinical 
neuropsychologists  in  accurately  identifying  individuals  with  brain  dysfunction. 


INTRODUCTION 
The  Problem  of  Traumatic  Brain  Injury 
Traumatic  brain  injury  (TBI)  is  a  physiological  disruption  of  brain  function  that 
results  from  the  head  being  struck,  the  head  striking  an  object,  or  the  brain  undergoing  an 
acceleration/deceleration  movement  (e.g.,  whiplash)  without  direct  trauma  to  the  head 
(American  Congress  of  Rehabilitation  Medicine  [ACRM],  1993).  There  are 
approximately  1.5-2  million  new  cases  of  TBI  in  the  United  States  each  year  (National 
Institute  of  Health,  1998),  and  prevalence  estimates  suggest  that  approximately  5.3  million 
Americans  (2  percent  of  the  population)  live  with  TBI-related  deficits  today. 
Epidemiological  evidence  suggests  that  the  annual  incidence  of  TBI  (200/100,000;  Kraus, 
1993)  exceeds  that  of  other  serious  neurological  diseases  such  as  multiple  sclerosis 
(7.5/100,000;  Mayr  et  al.  2003),  Parkinson's  disease  (20/100,000;  Fahn,  1995)  and 
Alzheimer's  disease  (123/100,000;  Schoenberg,  Kokmen,  &  Okazaki,  1987)  combined. 
Adolescents,  young  adults,  and  individuals  over  age  75  are  at  greater  risk  for  sustaining 
TBI,  and  males  are  two  times  more  likely  to  be  injured  than  females.  Forty-nine  percent 
of  TBI's  are  transportation  related  (e.g.,  involving  motor  vehicles,  bicycles,  pedestrians, 
and  recreational  vehicles),  and  26  percent  are  due  to  falls.  Firearms  incidents  account  for 
10  percent  of  all  TBI's,  and  assaults  account  for  8  percent.  The  residual  7  percent  are  due 
to  "other"  causes  (e.g.,  falling  objects;  Thurman,  Alverson,  Dunn,  Guerrero,  &  Sniezek, 
1999;  Thurman,  2001).  From  an  economic  perspective,  the  direct  costs  of  medical  care 
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($4.5  billion),  injury-related  work  loss  and  disability  ($20.6  billion),  and  lost  income 
resulting  from  premature  death  ($12.7  billion)  contribute  to  an  estimated  annual  cost  of 
$38  billion  dollars  in  the  United  States  alone  (Max,  MacKenzie  &  Rice,  1991).  After 
adjusting  for  inflation,  theses  costs  are  approximately  $48.3  billion  dollars  annually 
(Lewin,  1992). 

The  symptoms  following  TBI  are  often  devastating  and  mvolve physical  (e.g., 
nausea,  vomiting,  dizziness,  headache,  blurred  vision,  sleep  disturbance,  quickness  to 
fatigue,  lethargy,  or  sensory  loss),  cognitive  (e.g.,  involving  attention,  working  memory, 
learning,  speech/language,  or  executive  ftinctions),  and  behavioral  emotional  (e.g., 
irritability,  quickness  to  anger,  disinhibition,  or  emotional  lability;  ACRM,  1993)  systems. 
Susceptibility  of  the  Frontal  Lobes  in  TBI 

Relative  to  other  areas  of  the  cerebrum,  the  frontal  lobes  and  polar  areas  of  the 
temporal  lobes  are  disproportionately  susceptible  to  damage  following  TBI  (Bigler,  1999). 
Their  proximity  to  the  frontal  plate  of  the  skull  and  the  protuberances  of  the  floor  of  the 
anterior  cranial  fossa  (e.g.,  cribriform  plate),  and  the  medial  temporal  fossa,  respectively, 
compounds  their  vulnerability  to  deceleration  injuries  with  blunt  head  trauma  (e.g.,  motor 
vehicle  accidents,  fall).  This  is  evidenced  by  the  high  frequency  of  cases  with  acute 
cerebral  contusions  and  hematomae  in  these  regions  following  TBI  (Benson  &  Miller, 
1999;  Stuss  &  Benson,  1986;).  Follow-up  neuroimaging  studies  attempting  to  elucidate 
the  resolution  of  these  lesions  over  time  have  found  that  chronic  lesions  eventually  become 
hypodense,  suggestive  of  encephalomalacia  and  subsequent  fimctional  loss  (Benson  & 
Miller,  1999).  When  this  type  of  pathology  involves  the  frontal  lobes,  fimctional  deficits  in 
behavior  and  emotion  (e.g.,  impulsivity,  facetiousness,  apathy  and  behavioral  regulation; 


Benson  &  Miller,  1999;  Blumer  &  Benson,  1975;  Damasio  &  Anderson,  1993;  Lhermitte, 
Pillon,  &  Serdaru,  1985;  Malloy,  Bihrle,  Duffy,  &  Cimino,  1993;  Sarazin  et  al.,  1998)  and 
cognition  (e.g.,  sequencing,  abstract  thinking,  planning,  inhibition,  and  working  memory; 
Anderson,  Bigler,  &  Butler,  1995;  Brown  &  Levin,  2001;  Kimberg,  D'Esposito,  &  Farah, 
1999;  Lezak,  1995;)  may  follow.  Behavior,  emotion,  and  cognition  collectively  contribute 
to  executive  functioning  which,  according  to  Lezak  (1995),  is  "the  capacity  that  enables  a 
person  to  engage  successfully  in  independent,  purposive,  self-serving  behavior"  (p.  42). 
Deficits  in  executive  functioning  (executive  dysfunction)  have  been  referred  to  as  the 
"dysexecutive"  (Wilson,  Evans,  Emslie,  Alderman,  &  Burgess,  1998)  or  "frontal  lobe" 
syndrome  (Malloy  et  al.,  1993),  but  some  researchers  (Bigler,  1988;  Stuss  &  Alexander, 
2000)  have  cautioned  against  the  notion  of  describing  a  syndrome  based  on  its  anatomical 
locus. 

Although  the  frontal  lobes  and  temporal  poles  are  most  susceptible  to  damage 
following  TBI  (Bigler,  1999),  the  effect  of  rapid  inertial  changes  to  the  head  or  body 
sometimes  results  in  diffuse  pathology  throughout  the  cerebrum.  In  injuries  that  include 
blunt  head  trauma,  the  site  of  impact  to  the  skull  (e.g.,  the  frontal  plate  when  striking  the 
windshield  in  an  MVA)  represents  the  "coup"  site  and  is  the  location  where  brain  damage 
is  usually  maximal.  The  site  directly  opposite  from  the  coup  location  (e.g.,  the  occipital 
pole)  is  the  "contra-coup"  site.  Significant  pathology  may  occur  at  the  contra-coup  site 
due  to  the  brain  retracting  and  striking  the  occiput  following  the  initial  rapid  deceleration 
of  the  skull.  Massive  negative  pressure  gradients  resulting  from  the  initial  impact  also 
contribute  to  damage  at  the  contra-coup  site.  Contusions,  sub-  and  epi-dural  hematomae, 
intracerebral  hemorrhages  and  primary  axotomy  are  all  examples  of  the  primary  injury  in 


TBI.  Secondary  injury  includes  intracellular  cytotoxicity,  herniation,  ischemia  and 
secondary  axotomy  that  may  differentially  affect  outcome. 

Because  of  the  diffuse  and  heterogeneous  nature  of  the  primary  injury  and  the 
unique  secondary  injuries  that  are  unique  in  each  case,  TBI  patients  have  not  generally 
been  considered  ideal  candidates  for  discrete  structure/function  studies  (Damasio  & 
Damasio,  1999;  Damasio  &  Geschwind,  1985;  Damasio  &  Damasio,  1989).  However, 
their  proclivity  for  sustaining  prefrontal  pathology  suggests  that  they  may  be  good 
candidates  for  studying  dysexecutive  behavior. 
Models  of  Executive  Functioning. 

Attempts  to  elucidate  the  nature  of  executive  functioning  have  been  made  by 
neuropsychologists,  physiological  psychologists  and  most  recently  cognitive  psychologists. 
Early  explanations  based  on  neuropsychological  understanding  of  prefrontal/frontal  lobe 
functioning  were  espoused  by  Luria  (1973).  According  to  Luria,  the  brain  is  organized 
hierarchically  into  three  functional  units.  The  first  fianctional  unit  provides  "regulating 
tone  and  waking  and  mental  states"  (p.  44)  and  involves  the  ascending  and  descending 
reticular  activating  systems.  The  second  functional  unit  is  involved  in  receiving,  analyzing 
and  storing  information  and  involves  primary,  secondary,  and  tertiary  cortical  zones. 
Medial-temporal  lobe  structures  are  also  part  of  this  unit.  The  final  functional  unit,  which 
sits  atop  the  neuraxis  in  the  position  of  the  fi-ontal  lobes,  programs,  regulates,  and  verifies 
activity  and  involves  the  frontal  lobes.  This  unit  performs  executive  functioning  and  is 
highly  reliant  upon  the  integrity  of  the  prefi^ontal  cortices  for  performance.  Although 
Luria  provided  an  integrated  neuropsychological  explanation  of  executive  behavior,  other 


researchers  provided  more  specific  hypotheses  about  the  hierarchical  nature  of  cognitive 
functioning. 

Following  Shiffrin  and  Schneider's  (1977)  seminal  paper  on  automatic  and 
controlled  processes,  Shallice  and  Evans  (1978)  and  Shallice  (1982)  proposed  a 
hierarchical  model  of  behavior  that  included  executive/control  processes.  Shallice' s 
information  processing  model  was  comprised  of  cognitive  units,  schemas,  contention 
scheduling,  and  a  supervisory  attentional  system  (SAS).  Shallice  referred  to  cognitive 
units  as  modules  of  cognitive  processes  (e.g.,  memory,  language)  that  were  generally 
represented  by  specific  anatomical  pathways  (e.g.,  medial  temporal  lobe,  perisylvian 
region).  Schemas  refer  to  behaviors  that  are  dependent  upon  multiple  cognitive  units; 
however,  schemas  that  are  routinized  and  overleamed  exist  for  activities  that  place  a 
minimal  load  on  conscious  behavior  such  as  throwing  a  spiral  football  pass  for  an 
experienced  quarterback  or  riding  a  bicycle  for  an  expert  cyclist.  These  complex  yet 
routinized  behaviors  place  a  minimal  load  on  an  organism's  attentional  capacity. 
Contention  scheduling  refers  to  the  selection  of  a  specific  set  of  schemas  in  order  to  meet 
the  organism's  demands  within  the  environmental  milieu.  For  example,  contention 
scheduling  is  required  when  one  negotiates  a  familiar  library's  organizational  system  in 
order  to  check  out  a  book.  Finally,  the  SAS  organizes  and  manages  all  responses  while 
negotiating  nonroutine  tasks.  The  SAS  is  especially  activated  when  contention  scheduling 
does  not  work  or  when  the  solution  to  an  environmental  contingency  is  unknown. 
Shallice  posited  that  the  fi"ontal  lobes  perform  the  functions  of  the  SAS  and  that 
contention  scheduling,  but  not  the  SAS,  is  often  preserved  following  brain  injury.  In  this 
example,  the  flinctions  of  the  SAS  appear  synonymous  with  executive  control. 


Baddeley  (1986)  suggested  that  executive  processes  play  an  integral  role  in 
working  memory.  He  suggested  a  Multiple  Component  Model  of  working  memory  that 
included  two  highly  specific  slave  systems  responsible  for  the  processing  and  temporary 
maintenance  of  verbal  (e.g.,  Articulatory  Loop)  and  visuospatial  (e.g.,  Visuospatial 
Scratch  Pad)  material,  and  a  third  component  referred  to  as  the  Central  Executive.  The 
function  of  the  Central  Executive  is  to  control  and  regulate  the  working  memory  system 
and  is  thought  to  include  the  prefrontal  cortex. 

Models  of  frontal  lobe  functioning  from  animal  lesion  and  unit  recording  studies 
have  also  contributed  to  our  understanding  of  some  of  the  frontal  lobe  functions.  Fuster 
(2002)  reported  that  the  prefrontal  cortex  integrates  temporally  separate  units  of 
perception,  action  and  cognition  into  a  sequence  toward  a  goal.  He  suggested  that 
temporal  organization  (e.g.,  if  this  now,  then  that  later;  if  earlier  that,  then  now  this) 
occurred  by:  (a)  invoking  a  preparatory  set;  (b)  utilizing  working  memory;  and  (c) 
facilitating  inhibitory  control  of  interference,  and  that  temporal  organization  was  mediated 
by  the  prefrontal  cortex  and  was  executive  in  nature. 

Goldman-Rakic  (1987)  also  emphasized  the  role  of  the  prefi-ontal  cortex  in 
maintaining  internal  representations  across  time.  In  work  with  non-human  primates,  she 
showed  a  strong  relationship  between  activity  in  specific  neurons  in  the  prefrontal  cortex 
and  performance  on  delayed-response  tasks.  She  hypothesized  that  internal 
representations  guided  behavior  and  that  the  maintenance  of  these  representations  was 
performed  by  a  cortical  network  that  included  the  prefrontal  cortex.  Researchers  in  the 
area  of  clinical-cognitive  neuroscience  have  corroborating  evidence  fi"om  functional 
magnetic  resonance  imaging  (fMRI)  for  the  role  of  focal  prefrontal  activation  in  working 


memory  tasks  (Braver  et  al.,  1997;  Cohen  et  al.,  1997;  Curtis  &  D'Esposito,  2003; 
D'Esposito,  Postle,  &  Rypma,  2000) 

Cognitive  psychologists  have  emphasized  how  impaired  working  memory  impacts 
cognition  (Cohen  &  Servan-Schreiber,  1992).  The  authors  suggested  that  information- 
processing  deficits  in  patients  with  prefrontal  damage  may  be  caused  by  "a  degradation  in 
the  ability  to  construct  and  maintain  an  internal  representation  of  context"  (p.  46). 
Context  refers  to  information  held  in  mind  in  order  to  mediate  an  appropriate  behavioral 
response.  This  theory  was  based  on  previous  work  by  Fuster  (1980)  and  Goldman-Rakic 
(1987)  who  hypothesized  that  specific  cells  in  the  prefrontal  cortex  are  specific  to  a 
particular  stimulus  and  response  and  remain  active  during  the  delay  between  these.  Cohen 
and  Servan-Schreiber  (1992)  suggested  that  the  prefrontal  cortex  was  directly  involved  in 
maintaining  representations  that  inhibit  reflexive  or  habitually  reinforced  behaviors  to 
obtain  a  goal  (e.g.,  response  inhibition).  This  fimction  of  the  prefrontal  cortex,  to  maintain 
internal  representations  that  guide  behavior,  was  later  referred  to  as  cognitive  control 
(Braver,  Barch,  &  Cohen,  1999;  Miller  &  Cohen,  2001;). 

According  to  Miller  &  Cohen  (2001),  the  prefrontal  cortex  exerts  cognitive  control 
by  biasing  signals,  in  a  top-down  fashion,  in  those  parts  of  the  brain  responsible  for 
processing  sensory  (e.g.,  secondary  and  tertiary  regions  of  occipital,  temporal,  and  parietal 
cortices),  limbic  and  memory  retrieval  (e.g.,  direct  and  indirect  connections  with  the 
amygdala,  hypothalamus,  and  hippocampus),  and  motor  (e.g.,  supplementary  motor  area, 
pre-supplementary  motor  area,  premotor  area,  cerebellum,  superior  colliculus,  and  basal 
ganglia)  fiinctioning.  The  authors  state,  "The  aggregate  effect  of  these  bias  signals  is  to 


guide  the  flow  of  neural  activity  along  pathways  that  establish  the  proper  mappings 
between  inputs,  internal  states,  and  outputs  needed  to  perform  a  given  task"  (p.  171). 

The  context  maintenance  hypothesis  proposes  that  behavioral  inhibition  may 
reflect  the  operation  of  PFC-mediated  context  maintenance  processes,  and  that  working 
memory  and  response  inhibition  reflect  the  operation  of  a  single  underlying  processing 
mechanism.  In  this  context,  inhibition  refers  to  the  ability  to  inhibit  or  overcome  highly 
learned  or  otherwise  pre-potent  but  contextually  inappropriate  response  tendencies.  As 
previously  noted,  the  linking  of  PFC  representations  to  the  organization  of  prospective 
action  has  been  proposed  to  be  a  critical  fijnction  of  the  prefi-ontal  cortex  (Fuster,  1997; 
Goldman-Rakic,  1987),  and  preparation  to  overcome  contextually  inappropriate  responses 
is  an  important  component  of  this  system. 

The  aforementioned  models  of  executive  functioning/control  have  provided  a 
theoretical  fi-amework  for  the  development  of  psychological  tests  that  measure  some  of 
the  individual  component  processes  of  executive  functioning/control  (e.g.,  updating, 
context  maintenance,  inhibition).  Such  tests  have  been  developed  fi'om  the  information- 
processing  and  chronometric  perspectives  in  cognitive  psychology  and  provide 
computerized  assessment  of  specific  processes  and  yield  accuracy  and  reaction-time  data 
that  can  be  analyzed  in  light  of  predictions  derived  from  cognitive  models  of  executive 
function.  Given  this,  computerized  psychological  tests  which  have  been  developed  fi'om 
the  cognitive  psychological  firamework  may  be  better  able  to  parse  the  component 
processes  associated  with  executive  ftinctioning/control  and  subsequently  provide  a  more 
accurate  picture  of  the  specific  executive  deficit  that  occurs  after  TBI.  Compared  to 
traditional  neuropsychological  tests  of  executive  dysfianction,  computerized  psychological 


tests  developed  from  the  cognitive  psychological  framework  may  improve  diagnostic 
accuracy  and  aid  the  clinician  in  better  characterizing  the  exact  nature  of  executive 
ftmctioning/control  problems  that  may  occur  after  mild  TBI  (Brooks,  Fos,  Greve,  & 
Hammond,  1999). 

Cognitive  psychological  tests  have  not  been  widely  used  in  clinical  settings;  but 
they  have  been  used  to  study  executive  ftinctioning/control  and  working  memory  in  a  wide 
range  of  diseases  and  disorders.  For  example,  the  N-Back  task  (see  Methods  section 
below),  a  prototypical  working  memory  paradigm  that  taps  all  of  the  primary  components 
typically  considered  to  be  subsumed  by  working  memory:  maintenance  and  manipulation, 
with  manipulation  comprising  sequencing,  updating  and  temporal  tagging,  has  been  used 
in  studies  of  individuals  with  schizophrenia  (Jansma,  Ramsey,  Van  Der  Wee,  &  Kahn, 
2004;  Perlstein,  Dixit,  Carter,  Noll,  &  Cohen,  2003),  pediatric  TBI  (Levin  et  al.,  2004), 
muhiple  sclerosis  (Lengenfelder,  Chiaravalloti,  Ricker,  &  DeLuca,  2003;  Wishart  et  al., 
2004),  obsessive  compulsive  disorder  (Van  Der  Wee  et  al.,  2003),  Klinefelter  syndrome 
(Fales  et  al.,  2003),  and  HIV  (Hinkin  et  al.,  2002).  Another  cognitive  psychological  test, 
the  AX  version  of  the  original  Continuous  Performance  Test  (Rosvold,  Mirsky,  Sarason, 
Bransome,  &.  Beck,  1956;  AX-CPT),  taps  sustained  attention,  inhibitory  control,  and  the 
active  maintenance  component  of  working  memory,  and  has  been  used  in  studies  of 
neurologically  intact  individuals,  older  adults,  and  in  individuals  with  schizophrenia 
(Braver,  Cohen,  &  Barch,  2002).  The  cued  Stroop  task,  a  computerized  single-trial,  cued 
version  of  the  original  Stroop  test  (Cohen,  Barch,  Carter,  &  Servan-Schreiber,  1999),  taps 
the  ability  to  form  and  actively  maintain  context  representations  and  to  implement 
cognitive  control  more  explicitly  than  does  the  card  version  of  the  Stroop  test  and  has 
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been  used  in  studies  of  neurologically  intact  individuals,  and  individuals  with 
schizophrenia,  depression  (Cohen,  Barch,  Carter,  &  Servan-Schreiber,  1999),  and  TBI 
(Seignourel  et  al.,  2004).  The  computerized  Wisconsin  Card  Sorting  Test  (WCST; 
Kimberg,  D'Esposito,  &  Farah,  1997)  is  a  modified  version  of  the  traditional  WCST  that 
neuropsychologists  have  been  using  in  clinical  settings.  The  modified  version  purportedly 
places  a  greater  demand  on  cognitive  set  shifting  and  speed  of  information  processing  and 
has  been  used  in  neurologically  intact  individuals  taking  bromocriptine  -  a  dopamine 
receptor  agonist  (Kimberg,  D'Esposito,  &  Farah,  1 997). 

Although  the  aforementioned  cognitive  psychological  tests  offer  some  advantages 
over  traditional  tests  of  executive  functioning/control,  they  have  only  relatively  recently 
been  developed  and  are  not  yet  in  wide  clinical  use.  The  tests  have  also  yet  to 
demonstrate  adequate  reliability  and  validity,  and  have  not  been  compared  to  traditional 
neuropsychological  tests  with  regard  to  their  diagnostic  accuracy.  What  is  needed  is  a 
study  that  combines  the  use  of  cognitive  psychological  tests  with  traditional 
neuropsychological  tests  to  determine  which  tests  are  most  usefijl  in  identifying  individuals 
with  executive  dysfianction/dyscontrol. 
Reliability  and  Validity  of  Traditional  Executive  Functioning  Tests. 

Evaluating  the  cognitive  and  emotional  sequelae  following  central  nervous  system 
damage  constitutes  the  hallmark  function  of  the  clinical  neuropsychologist  (Eubanks, 
1997).  These  assessments  involve  collecting  data  by  medical  records  review,  conducting 
clinical  interviews  (with  patient  and  corroborating  others),  making  behavioral  observations 
during  the  evaluation,  and  performing  formal  neuropsychological  testing  (Lezak,  1995; 
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Vandeqjloeg,  2000).  This  process  is  complete  when  the  clinician  integrates  these  data  and 
communicates  the  findings  to  the  referral  source  (Meyer  et  al,,  2001). 

Assessing  neuropsychological  performance  through  the  use  of  objective 
standardized  tests  is  critical  to  making  accurate  diagnostic,  prognostic  and  treatment- 
related  decisions.  A  flmdamental  prerequisite  for  using  neuropsychological  tests  to 
identify  neurocognitive  and/or  neurobehavioral  dysfunction  is  that  the  instruments  have 
adequate  reliability  and  validity  (Anastasi,  1988).  In  the  domain  of  psychometric  testing, 
reliability  refers  to  the  degree  of  consistency  in  obtained  performance  scores  across  test 
administrations.  Although  no  psychometric  test  has  absolute  reliability,  there  should  be 
adequate  consistency  in  performance  across  conditions  to  ensure  that  the  observed  change 
in  test  performance  represents  change  in  the  construct(s)  the  test  purportedly  measures  as 
opposed  to  error  variance.   Validity  refers  to  the  degree  to  which  a  test  fulfills  the 
purposes  for  which  it  was  intended  and  includes  content,  criterion-related,  and  construct 
validity  (RetzIafF&  Gibertini,  2000).  For  example,  if  the  "Demery  Test  of  Marine 
Preparedness"  is  to  be  valid,  it  should  (1)  test  level  of  physical  fitness,  firearms 
proficiency,  and  knowledge  of  battle-field  tactics  (content  validity);  (2)  predict 
performance  on  an  objective  criterion  such  as  war-fighting  skill  (criterion-related  validity); 
and  (3)  measure  the  degree  to  which  the  test  measures  a  theoretical  construct  such  as 
"combat  effectiveness"  (construct  validity).  If  tests  are  not  reliable  and  valid,  they  may 
yield  inaccurate  information  and  thwart  diagnostic  accuracy. 

Traditional  tests  of  executive  functioning  generally  suffer  firom  low  reliability  and 
validity  (Bowden  et  al.,  1998;  Cripe,  1996;  Kafer  &  Hunter,  1997;  Miyake,  Emerson,  & 
Friedman,  2000;  Rabbitt,  1997).  Bowden  et  al.  (1998)  tested  the  alternate  form  reliability 
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of  the  Wisconsin  Card  Sorting  Test  (WCST)  in  a  sample  of  75  neurologically  normal 
college  students.  In  the  first  form,  the  order  of  sorting  was  performed  as  explained  in  the 
test  manual  (e.g.,  color,  then  form,  then  number).  In  the  alternate  form,  the  sorting  order 
was  changed  (e.g.,  form,  then  color,  then  number).  The  authors  reported  alternate-form 
reliability  coefficients  that  ranged  from  .28  for  the  number  of  "correct"  responses,  to  .60 
for  the  number  of  "categories"  achieved.  The  average  reliability  coefficient  across  all  six 
scores  was  .43.  Using  Cohen's  (1988)  assertion  that  correlation  coefficients  of 
approximately  .20,  .50  and  .80  are  small,  medium  and  large,  respectively,  the  results  fi-om 
this  study  indicate  that  the  alternate  forms  reliability  of  the  WCST  is  marginal  at  best.  In 
fact,  the  authors  caution,  "Since  clinicians  employing  a  test  with  low  reliability  are  likely  to 
be  fi"equently  misled  by  erroneous  information,  we  recommend  that  clinicians  should  not 
employ  the  Wisconsin  until  such  times  as  the  test  is  shown  to  be  of  adequate  reliability,  or 
is  improved  to  meet  acceptable  standards"  (p.  253).  While  the  authors  are  correct  in 
recommending  that  test  results  fi^om  the  Wisconsin  Card  Sort  be  used  cautiously  due  to 
low  reliability,  the  authors  may  have  gone  beyond  the  data  by  cautioning  against  the  use  of 
the  test  under  any  circumstances. 

In  the  WCST  manual  (Heaton,  Chelune,  Talley,  Kay,  &  Curtiss,  1993),  the  authors 
reported  one  paper  that  addressed  the  reliability  of  the  test.  Axelrod,  Goldman,  and 
Woodard  (1992)  conducted  two  experiments  that  assessed  the  intrascorer/intrarater 
reliability  of  the  WCST.  In  the  first  experiment,  they  reported  interscorer/intrascorer 
reliability  coefficients,  for  three  experienced  clinicians,  which  ranged  fi-om  .88  for 
nonperseverative  errors  to  .93  for  perseverative  responses.  In  a  second  experiment  using 
novice  scorers,  they  reported  good  to  excellent  interscorer  and  intrascorer  agreement  as 
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evidenced  by  reliability  coefficients  from  .75  to  .97.  Although  these  reliability  coefficients 
for  scoring  the  WCST  are  excellent,  no  evidence  is  reported  in  the  test  manual  regarding 
the  stability  of  performance  across  repeated  test  conditions  (e.g.,  test-retest  reliability). 

Miyake,  Emerson,  and  Friedman  (2000)  suggest  that  the  primary  reason  why 
executive  functioning  tests  have  insufficient  reliability  is  due  to  a  lack  of  novelty 
associated  with  many  of  these  tests  at  re-test.    Phillips  (1997)  stated  that  it  is  imperative 
for  any  test  of  frontal  lobe  functioning  to  be  novel  and  effortful.  The  need  for  novelty  is 
based  on  the  idea  that  when  a  series  of  novel  test  problems  are  negotiated  for  the  first 
time,  effortful  and  controlled  processing  (eg,  hypothesis  testing,  problem  solving, 
monitoring  and  updating)  must  be  applied  to  find  a  solution  to  the  problem.  Once  the 
solution  has  been  identified,  novelty  is  lost  and  automatic  processing  takes  over  and  the 
task  no  longer  becomes  a  test  of  executive  abilities.  Assuming  that  novelty  is  critical  to 
the  assessment  of  executive  functioning,  what  is  needed  is  a  test  that  is  novel  and  effortful 
across  test  administrations. 

Most  tests  of  executive  functioning  suffer  from  poor  construct  validity.  Given  the 
numerous  problems  defining  the  executive  functions,  research  in  the  area  of  the  construct 
validity  of  executive  functioning  has  been  justifiably  slow  (Miyake  et  al.,  2000).  For 
example,  the  WCST  has  been  purported  to  assess  executive  functioning;  however,  without 
an  adequate  definition,  it  is  difficult  to  understand  how  the  test  can  measure  it.  The  test  is 
said  to  specifically  measure  mental  set  shifting,  cognitive  flexibility,  problem  solving  and 
organizational  abilities.  The  degree  of  overlap  between  these  constructs  is  unknown  and 
without  adequate  study,  the  examiner  is  left  to  decide  what  the  test  measures.  These 
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issues  illustrate  some  of  the  psychometric  dilemmas  that  clinicians  face  when  assessing 
executive  fiinctioning. 
Issues  Related  to  Diagnostic  Accuracy  of  Executive  Functioning  Tests. 

Adequate  reliability  and  validity  of  psychometric  tests  are  central  concepts  in 
neuropsychological  assessment.  Once  these  properties  have  been  demonstrated  in  a  given 
test,  a  test's  ability  to  detect  brain  damage  and  any  specific  features  of  the  damage  (e.g., 
location)  is  examined.  These  functions  have  been  referred  to  as  a  test's  sensitivity  and 
specificity,  respectively.  Sensitivity  refers  to  how  well  a  test  detects  a  disorder  if  the 
person  actually  has  the  disorder,  and  specificity  refers  to  the  likelihood  that  the  test  will  be 
normal  if  the  person  does  not  have  the  disorder  (Retzlaff  &  Gibertini,  2000).  Adequate 
sensitivity  and  specificity  help  the  clinician  identify  "true  positives"  and  "true  negatives" 
among  a  population  of  unknowns.  This  approach  to  identifying  the  presence/absence  of  a 
disorder,  sometimes  referred  to  as  a  test's  "hit-rate,"  is  derived  fi-om  signal  detection 
theory.  In  psychometric  testing,  however,  indices  that  reflect  a  test's  capability  to 
accurately  perform  these  discriminating  functions  are  referred  to  as  the  test's  "operating 
characteristics"  (Retzlaff  &  Gibertini,  2000).  Operating  characteristics  include  positive 
and  negative  predictive  power.  Positive  predictive  power  (PPP)  is  the  ratio  of  true 
positive  cases  to  all  test  positives.  More  specifically,  it  refers  to  the  probability  that  an 
impaired  test  score  represents  a  person  with  the  disorder.  Negative  predictive  power 
(NPP)  refers  to  the  ratio  of  true  negatives  to  all  test  negatives,  and  represents  the 
probability  that  a  person  without  an  impaired  test  score  does  not  have  the  disorder 
(Elwood,  1993).  The  reciprocal  of  PPP  is  the  true  positive  rate,  and  the  reciprocal  of  NPP 
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is  the  false  negative  rate.  These  indices  allow  us  to  derive  probability  estimates  about  the 
likelihood  that  a  positive  test  finding  represents  a  disorder. 

Traditional  neuropsychological  tests  of  executive  functioning  are  marginally 
capable  of  differentiating  between  brain-damaged  and  non  brain-damaged  controls; 
however,  these  tests  have  not  consistently  shown  adequate  specificity  to  frontal  lobe 
damage  per  se  (Anderson,  Damasio,  Jones,  &  Tranel,  1991;  Anderson  et  al.,  1995; 
Axelrod  et  al.,  1996;  Costa,  1988;  Heaton  et  al.,  1993).  Given  that  most  executive 
functioning  tests  have  adequate  sensitivity  but  poor  specificity  and  low  positive  and 
negative  predictive  power,  these  tests  may  ultimately  fail  to  accurately  discriminate 
between  those  individuals  who  have  brain  damage  and  those  who  do  not.  Developing 
tests  with  adequate  sensitivity,  specificity,  positive  and  negative  predictive  power  for 
assessing  executive  fijnctioning  is  an  area  that  needs  fijrther  empirical  investigation. 
Purposes  of  the  Studv 

The  purposes  of  this  study  were  threefold:  (1)  we  wanted  to  determine  how 
different  groups  of  participants  (e.g.,  controls,  mild  TBI,  and  moderate/severe  TBI) 
performed  on  tests  of  executive  functioning;  (2)  we  wanted  to  identify  and  compare  the 
operating  characteristics  of  different  executive  control  tests  in  the  diagnosis  of  TBI  and 
determine  if  level  of  positive  predictive  power  differed  across  different  types  of  tests;  and 
(3)  we  wanted  to  develop  a  linear  combination  of  predictor  variables  that  could  predict 
group  membership.  Our  specific  aims,  and  hypotheses/predictions  for  this  study  were  as 
follows: 

Aim  I:  Our  first  aim  was  to  compare  performance,  on  a  broad  range  of  tests 
designed  to  assess  executive  functioning,  between  participants  who  have  sustained  a 
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documented  TBI  (mild  or  moderate/severe)  and  healthy  comparison  participants  who  have 
not  sustained  a  TBI. 

Hypothesis/prediction  I:  We  hypothesize  that  group  mean  performance  scores,  on 
all  individual  tests  designed  to  measure  executive  dysfunction,  will  differ  as  a  function  of 
group  membership  (e.g.,  control,  mild,  moderate/severe).  We  predicted  that  controls 
would  perform  best  and  mild  TBI  participants  would  perform  better  than  moderate/severe 
TBI  participants. 

Aim  II:  Given  the  construct  specificity  problems  of  traditional  neuropsychological 
tests,  our  second  aim  was  to  determine  if  "purer"  tests  (e.g.,  cognitive  psychological  tests) 
of  executive  dysfunction  would  yield  greater  power  in  predicting  the  presence  of  brain 
dysfunction  when  compared  to  traditional  neuropsychological  tests  or  tests  designed  to 
assess  everyday  executive  dysfunction.  Our  goal  was  to  define  the  operating 
characteristics  of  each  test  at  three  levels  of  impairment  and  then,  through  simple  visual 
analysis,  determine  if  positive  predictive  power  (PPP)  differed  as  a  function  of  test  type 
(e.g.,  cognitive,  neuropsychological,  test  of  everyday  executive  dysflinction). 

Hypothesis/prediction  II;  We  hypothesized  that  cognitive  psychological  tests 
would  yield  higher  PPP  because  they  are  thought  to  be  "purer"  measures  of  executive 
dysfunction  than  other  tests.  Specifically,  we  predicted  that  cognitive  psychological  tests 
of  executive  dysfunction  would  yield  greater  positive  predictive  power  than  traditional 
neuropsychological  tests  or  tests  of  everyday  executive  dysfunction  (e.g.,  Behavioral 
Assessment  of  the  Dysexecutive  Syndrome  subtests). 
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Aim  III:  Our  third  aim  was  to  determine  if  predictor  variables  from 
neuropsychological  tests  were  able  to  reliably  discriminate  group  membership  (e.g., 
control,  mild,  or  moderate/severe). 

Hypothesis/prediction  III:  We  hypothesized  that  a  weighted  linear  combination  of 
predictor  variables  would  be  useful  in  differentiating  between  groups.  For  reasons 
described  in  Aim  II  above,  we  predicted  that  the  best  weighted  linear  combination  of 
predictor  variables  that  could  discriminate  group  membership  would  include  a  greater 
number  of  cognitive  psychological  tests  than  traditional  neuropsychological  tests  or  tests 
of  everyday  executive  dysfiinction. 


METHODS 

Participants 

The  sample  consisted  of  46  participants  with  a  history  of  TBI  and  24  non-TBI 
controls  (N=70).  All  participants  were  respondents  from  flyers  posted  at  the  University  of 
Florida  Health  Science  Center  or  from  an  advertisement  that  appeared  in  the  Brain  Injury 
Association  of  Florida  newsletter.  All  respondents  underwent  a  brief  telephone  screening 
to  ensure  that  they  met  the  basic  inclusion  criteria  as  either  a  TBI  or  control  participant. 
Potential  participants  were  excluded  from  the  study  if  they  had  previously  been  diagnosed 
with  serious  psychopathology  (e.g.,  schizophrenia,  bipolar  disorder),  attention  deficit 
disorder,  learning  disability,  or  reported  a  history  of  chronic  alcohol  or  drug  abuse.  Prior 
to  actively  participating  in  the  study,  respondents  were  asked  to  bring  medical  records 
with  them  to  the  study  if  they  had  access  to  them.  All  participants  provided  written 
informed  consent  according  to  procedures  established  by  the  Health  Science  Center 
Institutional  Review  Board  at  the  University  of  Florida.  All  individuals  were  paid  $50.00 
for  their  participation. 

TBI  individuals  were  classified  as  having  sustained  a  TBI  as  defined  by  the 
American  Congress  of  Rehabilitation  Medicine  (ACRM,  1993).  Specifically,  any 
mechanically  induced  disruption  of  brain  function,  as  evidenced  by  (a)  any  period  of  loss 
of  consciousness;  (b)  any  retrograde  or  anterograde  amnesia;  (c)  any  alteration  in  mental 
status  immediately  following  the  injury  (e.g.,  feeling  dazed,  disoriented  or  confused);  or 
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focal  neurological  sign(s)  constituted  a  positive  history  of  brain  injury.  Brain  injury 
severity  (e.g.,  mild,  moderate,  severe)  was  determined  by  reviewing  medical  records 
information  regarding  neurological  status  at  the  time  of  injury  such  as:  presence/absence 
and  duration  of  loss  of  consciousness  (LOC),  Glasgow  Coma  Scale  (GCS)  score, 
neuroradiological  reports,  and  other  pertinent  information  (e.g.,  neurological  examination, 
neuropsychological  reports). 

Seven  TBI  participants  (1  mild,  1  moderate,  and  5  severe  TBI)  brought  medical 
records  with  them  to  the  study.  Four  mild  TBI  participants  did  not  go  to  the  emergency 
room  after  their  injuries.  One  of  those  4  was  assessed  by  an  athletic  trainer  and  another 
participant  sought  outpatient  treatment  from  their  primary  care  physician  for  post- 
traumatic headache.  For  the  remaining  participants  who  were  admitted  to  the  emergency 
room  after  their  injury  and  who  did  not  bring  medical  records  to  the  study  (n=35),  brain 
injury  severity  was  determined  retrospectively  from  data  collected  by  a  semi-structured 
interview  (see  Appendix)  that  was  completed  with  the  participant  and,  if  present,  a 
collateral  informant.  For  the  thirty-five  participants  who  did  not  bring  medical  records  to 
the  study,  information  regarding  the  presence/absence  and  duration  of  LOC  and/or  PTA 
that  was  obtained  through  interview  was  used  for  injury  severity  classification.  The 
reliability  and  validity  of  retrospective  assessment  of  brain  injury  severity  based  on  this 
approach  has  been  demonstrated  by  past  research  (King,  Crawford,  Wenden,  Moss,  Wade, 
&  Caldwell,  1 997;  McMillian,  Jongen  &  Greenwood,  1 996),  and  the  method  of 
retrospective  questioning  used  in  this  study  was  based  on  the  methods  described  by  King 
et  al.  (1997).  Patients  were  asked  to  recall  as  much  as  they  could  remember,  in 
chronological  order,  about  what  happened  immediately  before  and  after  their  injury.  They 
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were  reminded  to  state  only  what  they  personally  could  remember  and  not  what  they  had 
come  to  learn  from  others  since  the  accident.  The  interviewer  required  lucid  recall  of 
material,  excluding  any  "islands  of  memory"  (Russell,  1971)  until  such  time  that  the 
participant  was  able  to  describe  a  continuous  stream  of  memories.  Although  the  method 
of  retrospective  interviewing  to  determine  brain  injury  severity  was  used  in  the  present 
study,  there  are  limitations  to  this  approach.  There  is  an  assumption  that  the  information 
the  patient  tells  the  examiner  about  their  injury  (e.g.,  what  happened,  who  was  there, 
where  the  event  occurred,  what  the  last  memory  was  before  accident,  what  the  first 
memory  was  after  accident,  etc.)  is  fi^ee  from  factual  material  they  learned  from  others 
(e.g.,  family,  friends,  medical  personnel).  Another  limitation  is  that  the  duration  of  coma 
is  added  to  the  duration  of  PTA  as  an  index  of  brain  injury  severity.  Patients  are  at  times 
intentionally  put  into  a  coma,  via  pharmacological  agents,  to  prophylactically  manage 
intracranial  swelling.  If  this  is  the  case,  the  duration  of  PTA,  and  subsequently  brain  injury 
severity,  may  be  artificially  inflated  in  some  cases.  Given  these  limitations,  external 
validation  of  injury  severity  classification  should  be  sought  when  relying  on  retrospective 
interview  alone. 

In  order  to  determine  if  retrospective  interviewing  alone  was  a  valid  method  for 
determining  injury  severity,  we  compared  injury  severity  classification  (e.g.,  mild, 
moderate,  or  severe)  obtained  through  retrospective  interviewing  to  injury  severity 
classification  obtained  through  medical  records  review  for  the  seven  TBI  participants  who 
brought  their  medical  records  to  the  study.  Given  the  post-hoc  nature  of  this  analysis,  the 
researcher  who  performed  the  comparison  was  not  blinded  to  injury  severity  at  the  time  of 
the  medical  records  review.  Subsequent  comparison  showed  6/7  participants  had  been 
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correctly  classified.  One  case  that  had  previously  been  classified  a  severe  TBI  became  an 
unconfirmed  severe  after  it  was  found  that  positive  neuroradiological  findings  were  the 
only  medical  records  available  for  this  participant.  Based  on  the  previous  work  of  King  et 
al,  (1997)  and  the  present  findings,  we  concluded  that  our  method  of  retrospective 
interviewing  was  a  valid  method  of  determining  brain  injury  severity. 

Once  all  injury-related  information  was  obtained,  TBI  participants  were  classified 
as  having  sustained  either  a  mild,  moderate  or  severe  TBI.  Mild  TBI  was  operationalized 
as  an  initial  GCS  score  of  13  -  15  (Teasdale  &  Jennett,  1974),  duration  of  LOC  less  than 
30  minutes,  or  a  duration  of  PTA  less  than  24  hours  (ACRM,  1993);  moderate  TBI  was  a 
GCS  score  of  9  -  12,  duration  of  LOC  fi"om  3 1  minutes  -  6  hours,  or  duration  of  PTA  of  1 
-  7  days  (Lezak,  1995);  and  severe  TBI  v/as  a  GCS  score  of  less  than  9,  duration  of  LOC 
greater  than  6  hours,  or  duration  of  PTA  greater  than  7  days.  Finally,  each  participant 
was  assigned  to  either  a  control  (n=24),  mild  TBI  (n^20),  or  moderate/severe  TBI  (n=26) 
group. 
Procedure 

All  testing  was  completed  at  the  Evelyn  F.  and  William  L.  McKnight  Brain 
Institute  of  the  University  of  Florida.  Following  the  aforementioned  informed  consent 
procedure,  all  participants  underwent  a  45-minute  semi-structured  clinical  interview  to 
collect  relevant  demographic  data  and,  in  the  case  of  the  TBI  group,  information  related  to 
the  nature  of  their  injury.  All  participants,  and  informants  if  present,  completed  "self  and 
"informant"  versions  of  the  modified  Neurobehavioral  Rating  Scale  -  Revised  (NRS-R; 
Mathias  &  Coats,  1999)  and  Dysexecutive  Questionnaire  (DEX;  Wilson  et  al.,  1998).  For 
participants  who  arrived  alone  for  the  study,  blank  "informant"  rating  forms  and  a  pre- 
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addressed  stamped  envelope  were  provided  and  the  participant  was  asked  to  have 
someone  who  could  document  their  everyday  fianctioning  complete  and  mail  back  the 
forms  to  the  study  coordinator.  Frequency  count  indicated  that  59/70  (84%)  informant 
reports  for  the  DEX  and  modified  version  of  the  NRS-R  were  obtained. 

After  all  forms  had  been  completed,  participants  underwent  a  4-5  hour  testing 
session.  To  ensure  that  optimal  performance  was  maintained  throughout  testing, 
participants  were  observed  for  signs  of  excessive  fatigue  (e.g.,  falling  asleep,  carelessness, 
boredom)  and  were  afforded  a  brief  rest  period  of  5-minutes,  if  necessary. 

Tests  were  completed  in  a  quasi-randomized  fashion.  All  "computerized"  tests 
(e.g.,  N-Back,  AX-CPT,  AX-CPT  Interference,  Cued  Stroop)  except  the  WCST 
(Computer)  were  randomized  and  performed  in  one  testing  "block."  All  other  tests  (e.g., 
paper-pencil)  were  randomized  and  completed  in  a  different  testing  "block."  The  only 
computerized  test  that  was  completed  during  the  paper-pencil  block  was  the  WCST 
(Computer).  Given  the  variation  in  administration  instructions  in  the  WCST  (Computer), 
the  WCST  (Card)  immediately  preceded  the  WCST  (Computer)  for  all  participants.  Type 
of  tests  administered  first  (e.g.,  computer,  paper-pencil)  was  counterbalanced  between 
participants  to  control  for  order  effects. 
Measures/Apparatus 

Most  tests  are  touted  as  measures  of  executive  control  (Table  1).  Other  tests  (e.g., 
PASAT,  Digit  Symbol)  were  administered  for  comparison  against  the  current  battery  of 
executive  control  tests  because  these  tests  are  highly  sensitive  to  brain  damage  (Cicerone 
&  Azulay,  2002;  Gronwall  &  Wrightson,  1 98 1 ;).  Raw  scores  from  all  tests  were 
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converted  to  standardized  scx)res  (Z-scores)  based  on  means  and  standard  deviations 
obtained  by  the  non-TBI  control  group  in  this  sample. 
Computer-based  cognitive  psychological  tests 

N-Back.  The  N-Back  task  is  a  sequential  letter  memory  task  that  parametrically 
varies  working  memory  load  across  0-,  1-,  2-  and  3 -back  conditions.  In  the  0-back 
condition,  the  subject  makes  a  button  press  in  response  to  a  letter  (target)  that  matches  a 
pre-specified  letter  (e.g.,  "X").  In  the  1-back  condition,  the  target  becomes  any  letter  that 
was  identical  to  the  one  immediately  preceding  it.  In  the  2-  and  3 -back  conditions,  the 
target  becomes  any  letter  identical  to  the  one  presented  two  and  three  trials  back, 
respectively.  Stimulus  encoding  and  response  demands  are  constant  across  conditions. 
Only  requirements  to  maintain  and  update  increasingly  greater  amounts  of  information  at 
higher  loads  differ.  This  task  may  be  considered  a  prototypical  working  memory  paradigm 
in  that  it  taps  all  of  the  primary  components  typically  considered  to  be  subsumed  by 
working  memory:  maintenance  and  manipulation,  with  manipulation  comprising 
sequencing,  updating  and  temporal  tagging  (Perlstein  et  al.,  in  press).  The  dependent 
variables  for  this  task  are  the  proportion  of  errors  (e.g.,  total  errors/total  trials)  on  trials  1 
and  2  (Nback  Low)  and  trials  3  and  4  (Nback  High). 

Stroop  (computer).  This  is  a  modification  of  Golden's  Stroop  task  (Golden,  1978) 
where  stimuli  vary  in  two  dimensions  (e.g.,  color  words  displayed  in  color)  and 
participants  are  instructed  to  respond  to  one  of  the  two  dimensions  (read  the  word,  or 
name  the  color).  The  context  provided  by  the  task  instruction  (e.g.,  name  the  color)  must 
be  used  to  override  the  influence  of  the  stronger  dimension  (e.g.,  read  the  word)  when  the 
task  is  to  respond  to  the  less  prepotent  or  automatic  dimension  (e.g.,  name  the  color). 
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Context  provided  by  the  task  instructions  must  be  used  to  overcome  this  influence  of  the 
stronger  dimension  when  the  task  is  to  respond  to  the  weaker  one  (Cohen,  Dunbar,  & 
McClelland,  1990).  However,  traditional  Stroop  experiments  do  not  usually  place  strong 
demands  on  the  ability  to  hold  information  in  mind  in  order  to  mediate  an  appropriate 
behavioral  response  (context  maintenance;  Perlstein,  Carter,  Barch  &  Baird,  1998).  This 
is  because  trials  are  almost  always  blocked  by  task  (i.e.,  word  reading  or  color  naming),  so 
that  the  same  instructions  (context)  are  repeatedly  reinforced.  However,  reliance  on 
context  could  be  increased  by  (a)  varying  the  task  to  be  performed  on  each  trial  and  (b) 
introducing  a  delay  between  the  task  instructions  for  each  trial  and  the  stimulus  to  be 
responded  to.  We  predicted  that  such  manipulations  should  further  increase  the  sensitivity 
of  this  task  to  deficits  among  patients  with  traumatic  brain  injury. 

At  the  beginning  of  each  trial,  participants  heard  an  instructional  cue  (the  word 
"color"  or  "word"  presented  auditorily  by  computer)  followed  by  a  stimulus  presented  on 
the  monitor.  Participants  were  instructed  to  respond  verbally  to  the  stimulus,  as 
designated  by  the  cue,  as  quickly  and  accurately  as  possible.  Each  stimulus  remained  on 
the  screen  until  the  participant  responded.  Reaction  times  (RTs)  were  automatically 
recorded  by  a  voice-activated  relay  connected  to  the  computer,  and  the  examiner  manually 
coded  response  accuracy.  Although  rare,  on  the  occasion  that  the  voice-activated  relay 
picked-up  irrelevant  words  or  noises  and  then  automatically  proceeded  to  the  next  trial, 
the  examiner  noted  the  trial  number  and  flagged  the  trial  as  invalid.  Invalid  trials  were 
excluded  from  later  statistical  analyses.  Two  interstimulus  intervals  between  the  cue  and 
the  onset  of  the  stimulus  were  used:  1 -second  and  5-seconds,  Three  colors  and  color 
words  were  used  (red,  green,  and  blue),  presented  in  each  of  three  conditions  (congruent. 
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neutral,  incongruent).  Congruent  stimuli  consisted  of  one  of  the  three  color  names 
presented  in  its  own  color.  Incongruent  stimuli  consisted  of  a  color  name  presented  in  one 
of  the  two  remaining  colors.  Neutral  stimuli  were  four  colored  XXXXs  for  color  naming 
trials  and  color  words  displayed  in  black  for  word  reading  trials.  A  total  of  180 
experimental  trials  were  distributed  equally  across  tasks  (color  naming,  word  reading),  cue 
stimulus  ISI  (short,  long),  and  conditions  (congruent,  neutral,  incongruent)  and,  within 
conditions,  across  the  different  stimuli  for  that  condition.  Trials  were  pseudo  randomly 
ordered  for  each  participant  with  the  constraint  that  each  trial  type  occurred  twice  in  every 
24  trials.  The  dependent  variable  for  this  task  is  the  proportion  of  errors  during  the  color 
incongruent  trials  for  long  (CL  Incongruent)  and  short  (CS  Incongruent)  ISI  trials. 

AX-Continuous  Performance  Test  (CPT).  In  this  version  of  the  CPT,  randomly 
chosen  letters  are  presented  sequentially  in  a  visual  display  and  participants  are  instructed 
to  respond  to  the  letter  "X"  only  when  it  follows  an  "A."  Performance  on  this  task  relies 
on  the  representation  and  maintenance  of  context  information,  insofar  as  the  correct 
response  to  "X"  depends  on  the  previous  stimuli  (A  or  non-A).  We  used  a  version  of  this 
task  that  we  modified  in  two  ways  (Servan-Schreiber,  Cohen,  &  Steingard,  1997).  First, 
we  increased  the  fi^equency  of  target  (A-X)  sequences  to  70%  of  trials,  with  the  remaining 
30%  divided  evenly  between  three  distracter  conditions:  a  non-A  followed  by  an  X 
(which  we  referred  to  as  the  BX  condition);  an  A  followed  by  a  non-X  (AY);  and  a  non-A 
followed  by  a  non-X  (BY).  We  hypothesized  that  this  fi^equency  distribution  would 
induce  a  strong  tendency  to  respond  to  X,  since  that  is  the  correct  response  for  87.5%  of 
the  trials  in  which  X  appeared.  This  tendency  would  have  to  be  inhibited  in  the  BX 
condition,  relying  on  the  use  of  context  (non-A  stimulus)  to  overcome  the  tendency  to 
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respond  to  X.  Thus,  in  our  variant  of  the  task,  a  failure  to  process  context  can  be  indexed 
not  only  by  misses  in  the  AX  (target)  condition,  but  also  by  false  alarms  in  the  BX 
condition.  Our  second  manipulation  was  to  vary  the  delay  (1 -second  or  5-seconds) 
between  the  cue  (A/non-A)  and  the  probe  (X/non-X)  stimuli  in  order  to  determine  if  errors 
increased  under  longer  delay  conditions.  The  5 -second  delay  is  believed  to  be  more 
difficult  because  it  places  a  greater  demand  on  the  ability  to  actively  maintain  context 
information  across  time.  Participants  responded  by  pressing  a  button  on  a  box  connected 
to  a  computer,  which  recorded  the  response  and  reaction  time  (RT).  Stimuli  appeared  for 
250  ms,  and  participants  had  a  total  of  1 -second  from  onset  in  which  to  respond, 
irrespective  of  interstimulus  interval  (ISI).  Distracter  stimuli  (non-A  and  non-X)  consisted 
of  the  remaining  letters  of  the  alphabet,  with  the  exception  of  K,  which  was  excluded 
because  of  its  similarity  to  X.  Feedback  was  provided  in  the  form  of  a  beep  for  correct 
responses,  a  buzzer  sound  for  false  alarms  or  misses  and  no  sound  for  a  correct  rejection. 
A  total  of  400  trials  were  run.  ISI  and  condition  were  varied  pseudorandomly  over  trials, 
with  the  constraints  that  50%  of  trials  occurred  at  each  ISI,  the  distribution  of  conditions 
was  the  same  for  each  ISI ,  and  each  trial  type  occurred  at  least  once  every  20  trials.  The 
dependent  variable  used  for  this  task  is  d'  context  (d'  AX  No  Interference  Long  Delay=d' 
AX  NoInt  Long)  which  is  calculated  using  AX  (target)  and  BX  (non-target)  trials  to 
provide  an  overall  measure  of  use  of  cue  information  to  disambiguate  probe  stimulus 
states. 

AX-CPT  Interference.  The  format  of  this  task  is  identical  to  the  AX-CPT  with  the 
exception  that  2  random  letters  are  presented  during  the  delay  between  the  cue  (A-non-A) 
and  the  probe  (X/non-X).  The  purpose  of  this  procedure  was  to  determine  if  the  presence 
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of  the  interfering  letters  degraded  context  maintenance.  Consistent  with  the  AX-CPT,  the 
dependent  variable  used  for  this  task  is  d'  context  (d'  AX  Interference  Long  Delay=AX 
Int  Long). 

Wisconsin  Card  Sorting  Test  (WCST-Computer).  This  task  is  a  computerized, 
speeded  version  of  the  WCST  developed  by  Kimberg,  D'Esposito,  and  Farah  (1997a). 
The  task  requires  participants  to  match  a  series  of  target  cards  presented  individually  in 
the  middle  of  the  screen  with  any  one  of  four  reference  cards  shown  near  the  top  of  the 
screen.  Participants  were  instructed  to  sort  the  target  cards  into  piles  under  the  reference 
cards  according  to  one  of  three  categories  or  stimulus  attributes-color,  number,  or  shape. 
Participants  were  also  told  that  only  one  attribute  was  correct  for  each  target  card.  Each 
target  card  appeared  until  a  response  was  given  or  a  maximum  of  five  seconds  had 
elapsed,  at  which  point  the  next  trial  commenced  immediately.  Participants  receive  visual 
feedback,  (e.g.,  "right"  or  "wrong")  appearing  below  the  sorted  target  card,  based  on  the 
accuracy  of  their  response.  If  the  participant  does  not  categorize  the  target  card  within 
the  5-second  time  constraint,  the  phrase  "time  out"  appears  to  the  right  of  the  target  card 
in  the  ensuing  trial.  The  category  stays  the  same  until  the  participant  correctly  performs 
eight  consecutive  sorts,  at  which  time  the  sorting  criterion  changes.  The  participants  are 
initially  told  that  the  sorting  criterion  changes,  but  they  are  not  explicitly  told  the  exact 
number  of  correctly  sorted  cards  to  be  achieved  before  the  criterion  shifts.  After 
practicing  on  1 8  cards,  the  main  task  begins  and  continues  until  the  participant 
successfully  achieves  15  sorting  categories  or  the  total  number  of  target  cards  exceeds 
288.  The  method  of  modified  instructional  cues  at  the  beginning  of  the  task  has  proven 
usefial  in  distinguishing  concept  formation  versus  perseverative  responding  deficits  in  other 
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patient  populations  with  executive  dysfunction  (e.g.,  schizophrenia;  Goldman,  Axelrod,  & 
Tompkins,  1 992),  The  dependent  variables  on  this  task  are  the  proportion  of 
perseverative  errors  (e.g.,  total  perseverative  errors/total  number  of  trials;  Prop.  Persev. 
Errors),  and  the  total  number  of  categories  completed  (Cat.  Completed).  These 
dependent  variables  were  selected  based  on  recent  findings  indicating  that  these  variables 
most  accurately  discriminate  between  patients  with  frontal  versus  non-frontal  lobe  damage 
(Demakis,  2003). 
Paper-pencil  neuropsychological  tests 

Wisconsin  Card  Sorting  Test  (WCST-Card).  This  is  the  traditional  128-card 
version  of  the  WCST  (Heaton  et  al.,  1993).  In  this  task,  participants  sort  cards  to  one  of 
four  reference  cards  in  front  of  them  according  to  one  of  three  stimulus  attributes  -  color, 
number  or  shape.  Participants  were  not  told  beforehand  how  to  sort  the  cards  but  they 
were  provided  verbal  feedback  from  the  examiner  regarding  the  accuracy  of  their  sort, 
(e.g.,  "correct"  or  "incorrect").  The  category/stimulus  attribute  remained  constant  until 
ten  consecutively  cortect  sorts  are  made.  After  ten  correct  sorts,  the  category  criterion 
changes.  Each  criterion  (e.g.,  color,  number  &.  shape)  appeared  two  times  during  the  task 
for  a  total  of  six  criterion  changes.  The  test  was  concluded  when  6  categories  had  been 
achieve  or  when  128  trials  had  been  administered,  whichever  occurred  first.  The 
dependent  variables  are  the  proportion  of  perseverative  errors  (e.g.,  total  perseverative 
errors/total  number  of  trials;  Prop.  Persev.  Errors),  and  the  total  number  of  categories 
completed  (Cat.  Completed). 

Stroop  (Card).  The  Card  Stroop  (Golden,  1978)  is  comprised  of  three  conditions: 
word  reading,  color  naming,  and  color-word  naming.  In  the  word  naming  condition,  the 
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participant  is  instructed  to  read  the  words  on  the  stimulus  sheet  as  quickly  as  possible.  In 
the  color  naming  condition,  they  are  instructed  to  name  the  color  that  the  XXXX's  on  the 
stimulus  sheet  are  printed  in  as  quickly  as  they  can.  In  the  color-word  condition,  they  are 
instructed  to  name  the  color  of  the  ink  that  the  word  is  printed  in,  while  ignoring  what  the 
word  is,  as  quickly  as  they  can.  Each  trial  is  45  seconds  and  the  dependent  variable  is  the 
color-word  interference  (color  naming  minus  color  word  naming)  score  (Interference). 

Trail  Making  Test  (TMT).  The  Trail  Making  Test  (TMT;  Reitan  &  Wolfson, 
1985)  is  comprised  of  two  subtests:  Part-A  and  Part-B.  In  Part-A,  the  subject  is  required 
to  draw  lines  on  a  page  connecting  twenty-five  numbers  in  numerical  sequence  (e.g.,  1-2- 
3-4. . .)  as  quickly  as  possible.  In  Part-B,  the  subject  must  draw  lines  alternating  between 
numbers  and  letters  in  the  appropriate  sequence  (e.g.,  1-A-2-B-3-C. . . ).  Both  Parts-A 
and  -B  assess  rapid  visual  search  and  visuospatial  sequencing;  however,  Part-B  places 
additional  demands  on  cognitive-set  shifting  (Spreen  &  Strauss,  1998)  and  mental 
flexibility  (Bechtold,  Homer,  &  Windham,  2001).  The  dependent  variables  for  these  tasks 
are  the  total  number  of  seconds  to  complete  Part  A  (TMT  A)  and  Part  B  (TMT  B). 

Digit  Span  Test.  The  Digit  Span  test  (Wechsler,  1991)  is  comprised  of  two 
separate  elements:  digit  span  forward  and  digit  span  backward.  In  the  digit  span  forward 
subtest,  the  subject  is  verbally  presented  increasingly  longer  strings  of  single-digit 
numbers.  Trial  one  consists  of  two  digits  (e.g.,  7,3)  and  trial  seven  consists  of  9  digits 
(e.g.,  4,2,8,5,9,7,3,1,4).  In  the  digit  span  forward  task,  the  examiner  reads  the  numbers 
and  the  subject  repeats  them  back  to  the  examiner  in  the  same  order  in  which  they  were 
presented.  In  the  digit  span  backward  task,  trials  are  presented  as  they  were  in  the  digit 
span  forward  condition,  but  the  subject  must  repeat  the  numbers  backward  (e.g.,  3,  1,9  = 
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9,  1,  3).  A  two-trial  dyad  was  performed  for  each  set  of  digits  (e.g.,  set  of  two  digits  (trial 
1),  set  of  three  digits  (trial  2),  etc.).  For  each  set  of  digits,  scoring  is  assessed  in  the 
following  manner:  2-points  if  both  trials  are  passed;  1 -point  if  only  one  trial  is  passed;  and 
0-points  if  neither  trial  is  passed.  The  dependent  variables  are  the  total  number  of  points 
achieved  across  the  forward  (DSForward)  and  the  backward  (DSBackward)  trials. 

Paced  Auditory  Serial  Addition  Test  (PASAT).  The  PASAT  (GronwaU,  1977)  is  a 
test  of  information  processing  speed,  working  memory,  and  divided  attention.  Since  this 
test  has  been  demonstrated  to  be  very  sensitive  to  the  presence  of  brain  injury,  it  was 
included  in  the  battery  as  a  "gold  standard"  for  later  comparison.  We  employed  a  revised 
version  of  the  original  PASAT  (Gronwall,  1977)  described  by  Levin  et  al.,  (1987; 
PASAT-R),  which  presented  audio-taped  numbers,  ranging  from  1  to  9.  Participants  were 
instructed  to  add  sequential  numbers,  such  that  they  calculated  the  sum  of  each  of  the 
previous  two  numbers  they  heard.  They  provided  verbal  responses  indicating  their 
answers,  which  were  recorded  by  the  experimenter.  Participants  performed  four  trials  of 
the  PASAT-R,  differing  in  rate  of  stimulus  presentation,  in  the  sequence:  2.8s,  2.4s,  2.0s, 
and  1.6s.  Each  trial  contained  25  sequentially  presented  single-digit  numbers.  The 
dependent  variable  on  this  task  is  the  total  number  correct  across  all  trials  (PASAT  Total). 

Digit  Symbol  Test  The  Digit  Symbol  Test  (Wechsler,  1991)  consists  of  rows  of 
rectangles  where  the  top  half  of  the  rectangle  has  a  number  in  it  and  the  bottom  half  of  the 
rectangle  is  empty.  A  key  is  provided  at  the  top  of  the  page  with  numbers  (0-9)  in  the  top 
portion  of  the  rectangle  and  different  symbols  in  the  bottom  portion  of  the  rectangle. 
Each  number  is  associated  with  its  own  unique  symbol.  The  participant  is  to  insert  the 
appropriate  symbol  underneath  each  number  in  the  box  as  quickly  as  they  can.  Like  the 
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PAS  AT,  this  test  has  been  shown  to  be  very  sensitive  to  detecting  the  presence  of  brain 
damage  and  again  is  considered  a  "gold  standard"  in  TBI  assessment.  The  dependent 
variable  for  this  task  is  the  total  correct  (DSymbol)  in  120  seconds. 

Modified  version  of  the  Neurobehavioral  Rating  Scale  -  Revised  (NRS-R).  We 
used  a  modified  version  of  the  NRS-R  developed  by  Mathias  &  Coats  (1999)  that  has 
been  converted  from  a  clinician  rating  format  to  a  self-report  format  using  a  5-point  scale 
ranging  fi-om  "never"  (scored  0)  to  "very  often"  (scored  4).  Higher  scores  represent 
greater  difficulty  within  the  specific  cognitive  or  behavioral  domain.  Self-ratings  and 
independent  ratings  of  this  measure  were  used  to  increase  reliability  of  everyday 
functioning  assessment.  The  dependent  variable  used  on  each  measure  is  the  total  number 
of  points  on  the  self  (NRS  Self)  and  significant  other  (NRS  Other)  version  of  the  survey. 
Paper-pencil  tests  of  everyday  executive  dysfijnction 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome.  The  Behavioral 
Assessment  of  the  Dysexecutive  Syndrome  (BADS)  is  a  battery  of  subtests  that  may  be 
used  independently  to  collect  discrete  information  about  the  integrity  of  fi"ontaI  lobe 
functioning,  or  the  subtests  may  be  used  collectively  to  obtain  a  composite  score 
purportedly  indicative  of  gross  frontal  lobe  functioning  (Wilson  et  al.,  1998).  The 
dependent  variables  for  this  battery  are  the  total  raw  scores  for  each  subtest,  and  a 
composite  score  for  the  entire  battery.  Per  the  author's  direction,  the  "Temporal 
Judgment"  subtest  was  omitted  due  to  cultural  differences  between  the  normative  sample 
and  "American  users."  The  composite  score  was  the  sum  of  the  profile  scores  from  the 
remaining  subtests  (range  0-20;  TotalProScore).  Individual  subtests  of  the  BADS  battery 
are  described  below. 
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Rule  Shift  Cards.  This  test  requires  the  participant  to  respond  correctly  to  a  rule 
and  to  shift  from  one  rule  to  another.  A  book  of  playing  cards  is  placed  in  front  of  the 
subject  and  the  cards  are  turned  over  one  at  a  time  and  the  participant  says  "yes"  or  "no" 
according  to  a  rule  that  they  have  in  front  of  them.  In  the  first  trial,  the  task  rule  is  to  say 
"yes"  to  a  red  playing  card  and  "no"  to  a  black  playing  card.  In  the  second  trial,  the  task 
rule  changes  and  the  participant  says  "yes"  if  the  card  is  the  same  color  as  the  previous 
one  and  "no"  if  it  was  not.  The  task  is  thought  to  measure  response  inhibition  and  the 
ability  to  maintain  and  shift  cognitive  set.  The  dependent  variable  on  this  task  is  the  raw 
score  (RSCTotal)  which  is  the  total  number  of  errors  and  ranges  from  0-40. 

Action  Program  Test.  This  task  was  developed  to  provide  subjects  with  a  novel, 
practical  test  that  requires  a  5-step  plan  of  action  in  order  to  solve  the  problem.  The 
subject  is  presented  with  a  1-inch  thick  plastic  rectangular  stand.  At  one  end  of  the  stand 
is  a  large  transparent  bowl  that  is  two-thirds  fiill  of  water  and  is  covered  by  a  removable 
lid  with  a  small  hole  in  the  center.  A  glass  tube  with  a  small  cork  in  the  bottom  is  set  into 
the  other  end  of  the  stand.  To  the  left  of  the  stand  is  placed  a  metal  rod  which  is  not  quite 
long  enough  to  reach  the  cork,  and  a  small  plastic  tube  that  is  threaded  on  the  top.  A 
threaded  plastic  cap  that  fits  onto  the  threaded  tube  is  lying  next  to  the  plastic  tube. 
Participants  are  asked  to  get  the  cork  out  of  the  tube  using  any  of  the  objects  in  front  of 
them  but  without  lifting  up  the  stand,  the  tube  or  the  water  containing  bowl  and  without 
touching  the  lid  with  their  fingers.  The  plan  requires  the  subject  to  sequentially  progress 
through  the  five  stages.  If  any  stage  is  unsurpassable  by  the  subject,  the  examiner 
provides  a  prompt.  One  point  is  deducted  for  each  prompt  given.  There  is  no  time  limit 
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for  this  task.  The  dependent  variable  on  this  task  is  the  raw  score  (APRaw)  which  ranges 
from  0-5  and  is  based  on  the  number  of  problem  solving  steps  the  participant  completes. 

Key  Search  Test.  Subjects  are  presented  with  an  8  1/2  X  11  piece  of  white  paper 
with  a  6  X  6  inch  square  in  the  center  and  a  small  black  dot  3  inches  below  the  square. 
Subjects  are  told  to  imagine  that  the  square  is  a  large  field  in  which  they  have  lost  their 
keys.  They  are  asked  to  draw  a  line,  starting  on  the  black  dot,  to  show  where  they  would 
walk  to  search  the  field  to  make  absolutely  sure  they  would  find  their  keys.  Scoring  is 
based  on  the  adequacy  of  the  search  pattern  to  find  the  keys  with  elements  such  as 
entering  the  field  from  a  comer,  vertical/horizontal  searching  versus  concentric  patterns 
receiving  more  points.  The  dependent  variable  on  this  task  is  the  raw  score  (KSRaw) 
which  ranges  from  0-16. 

Zoo  Map  Test  In  this  test,  subjects  are  required  to  demonstrate  how  they  would 
visit  a  series  of  designated  locations  on  a  map  of  a  zoo.  However,  when  planning  the 
route,  certain  rules  must  be  adhered  to.  The  first  trial  consists  of  a  high-demand  version 
of  the  task  in  which  the  planning  abilities  of  the  subject  are  vigorously  tested  by  requiring 
the  subject  to  adhere  to  navigation  rules  while  walking  through  the  zoo  (e.g.,  must  start  at 
entrance,  finish  at  picnic  area,  can  not  use  shaded  pathways  more  than  once,  must  visit 
fixed  number  of  sites).  Errors  are  made  if  the  subject  does  not  plan  the  route  effectively. 
In  the  second  trial,  or  low-demand  trial,  the  subject  is  simply  required  to  follow  the 
instructions  to  produce  an  error-free  performance.  Directly  following  the  instructions  in 
the  high-demand  version  of  the  test  will  maximize  the  total  number  of  errors.  Thus,  in  the 
high  demand  version,  subjects  have  to  structure  their  own  plan  unaided  and  in  the  low- 


35 
demand  version  help  is  provided  wath  the  overall  structure.  The  dependent  variable  on  this 
task  is  the  overall  raw  score  (ZooTotal)  which  ranges  from  0-16. 

Modified  Six  Elements  Test.  This  is  a  simplified  version  of  the  original  Shallice 
and  Burgess  (1991)  test,  and  involves  the  participants  being  given  instructions  to  do  three 
tasks  (e.g.,  dictation,  arithmetic  and  picture  naming),  each  of  which  is  divided  into  two 
parts  referred  to  as  parts  "A"  and  "B."    Thus,  there  are  two  dictation  tasks,  two  sets  of 
arithmetic  problems,  and  two  sets  of  pictures  of  objects  that  have  to  be  named,  for  a  total 
of  six  subtasks  (elements).  The  participant  is  required  to  do  something  from  each  of  the 
six  subtasks  within  a  10-minute  period.  In  addition,  they  are  told  that  there  is  one  rule 
which  must  not  be  broken:  They  are  not  allowed  to  do  the  two  parts  of  the  same  task 
consecutively.  For  this  task,  it  is  not  how  well  the  participant  performs  in  the  individual 
components,  rather  the  test  measures  how  well  participants  organize  themselves.  Burgess 
(1997)  confirmed  that  the  Six  Elements  Test  makes  demands  on  a  subject's  ability  to  plan, 
organize,  and  monitor  behavior.  This  task  yields  variables  related  to  the  number  of 
subtasks  attempted,  the  number  of  times  the  rule  was  broken  for  each  type  of  task,  and  the 
maximum  amount  of  time  spent  on  any  one  subtask.  The  dependent  variable  on  this  task 
is  the  raw  score  (MSERaw)  which  ranges  from  0-6. 

The  Dysexecutive  Questionnaire  (DEX).  This  is  a  20-item  questionnaire  designed 
to  assess  a  wide  range  of  problems  in  patients  with  frontal  lobe  deficits.  The  questions 
sample  four  areas  of  fianctioning:  (a)  emotional  or  personality  fiinctioning,  (b)  motivation, 
(c)  behavioral  changes  and  (d)  cognitive  changes.  Each  item  is  scored  on  a  five-point 
likert  scale  (0-4)  based  on  the  frequency  that  the  symptom  occurs  (never-very  often).  One 
form  is  completed  by  the  subject  (self-rating)  and  another  form  is  completed  by  a 
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significant  other  (independent  rating)  who  has  contact  with  the  individual  each  day.  The 
raw  score  on  both  versions  (DEX  Self  and  DEX  Other)  ranges  from  0-80  and  serves  as 
the  primary  dependent  variable.  The  DEX  is  not  used  in  the  computation  of  the  BADS 
total  profile  score. 
Clinical  assessment 

The  Beck  Depression  Inventory  (BDI).  The  BDI  is  a  widely  recognized  measure 
of  depression  which  is  used  to  assess  the  extent  to  which  an  individual  currently  exhibits 
or  experiences  each  of  the  twenty-one  behaviors,  thoughts,  or  affective  features  commonly 
associated  with  depression  (Beck,  1972).  The  dependent  variable  for  this  task  is  the  total 
score  for  all  items. 

The  State-Trait  Anxiety  Inventory  (STAI).  Given  the  potential  confound  that 
anxiety  may  yield  on  executive  control,  the  STAI  -  form  Y  (Spielberger,  1977)  was 
administered  to  provide  information  about  the  presence  of  anxiety  at  the  time  of  testing 
(e.g.,  "state"  anxiety).  The  test  provides  information  about  temporary  conditions  of 
"state"  anxiety  versus  more  long-standing  conditions  of  "trait"  anxiety.  The  test 
specifically  addresses  feelings  of  apprehension,  tension,  nervousness,  and  worry.  The 
dependent  variable  is  the  total  score  for  the  "state"  anxiety  items. 

The  North  American  Adult  Reading  Test  (NAART).  The  NAART  (Blair  & 
Spreen,  1989)  is  a  reading  test  of  61  irregularly  spelled  words,  and  provides  an  estimate  of 
premorbid  intellectual  functioning.  The  participant  is  presented  with  the  word  sheet  and  is 
instructed  to  read  each  word.  The  dependent  variable  on  this  test  is  the  number  of  errors. 
The  error  score  is  used  in  a  formula  to  derive  an  estimate  of  full  scale  IQ  (FSIQ;  127.8  - 
.78[NAART  errors]). 


RESULTS 
The  collective  sample  (N=70)  was  comprised  of  61  (87%)  Caucasian,  4  (6%) 
African- American,  4  (6%)  Hispanic  and  1(1%)  Native- American  participants.  Control 
(n=24),  mild  TBI  (n=20)  and  moderate/severe  TBI  (n=26)  groups  were  compared  with 
univariate  ANOVAs  with  Bonferroni  corrections  on  basic  demographic  features  such  as 
age,  education,  and  NAART  FSIQ  (see  Table  2).  Analyses  revealed  no  differences 
between  groups  on  age  or  education;  however,  moderate/severe  TBI  participants  achieved 
a  significantly  lower  NAART  FSIQ  than  mild  participants.  No  association  between  group 
and  gender  was  found.  The  NAART  FSIQ  findings  notwithstanding,  these  groups 
appeared  essentially  equivalent  on  basic  demographic  features.  There  was  no  difference  in 
time  post-injury  between  mild  and  moderate/severe  TBI  groups,  but  as  expected 
significant  differences  were  observed  on  the  injury  severity  variable  'duration  of  PTA,' 
Mann-Whitney  U  (20, 17)  =  21 .00,  p  <  .001 .  Results  also  showed  that  moderate/severe 
TBI  participants  endorsed  greater  depressive  symptomatology  than  did  controls,  but  the 
intensity  of  depressive  symptomatology  did  not  meet  the  clinical  cutoff  score  for  clinical 
depression  recommended  by  Beck  (1987)  and  Marsella,  Sanborn,  Kamboka,  Shizuri,  & 
Brennan,  (1974).  Our  findings  of  depressive  symptom  endorsement  are  consistent  with 
similar  findings  in  the  TBI  literature  (Garske  &  Thomas,  1992).  On  the  STAI, 
moderate/severe  TBI  participants  endorsed  greater  State  anxiety  than  did  mild  TBI 
individuals,  but  the  intensity  of  the  anxiety  was  mild  (see  Table  2). 
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Table  2 

Between  group  comparisons  on  demographic  and  emotional  fiinctioning  variables 


Group  Assignment 


Controls  Mild  TBI         Mod/Sev  TBI 


24 


20 


26 


Age 


M 

35.13 

33.95 

40.73 

SD 

9,87 

10.49 

11.49 

Education 
M 

14.04 

15.03 

13.90 

SD 

1.71 

2.13 

2.10 

NAART  FSIQ 

M 

107.03 

109.20 

101.91 

SD 

7.33 

8.62 

11.18 

Gender  (%  M/F) 

50/50 

50/50 

65/35 

Time  Post-injury  (Months) 

M' 

N/A 

20.80 

25.58 

Duration  of  PTA  (Hours) 

M' 

N/A 

11.55 

27.76 

BDI 

M 

3.67 

6.15 

10.54 

SD 

4.14 

6.00 

8.61 

Mean  STAI  State 

M 

27.25 

26.67 

34.07 

SD 

5.23 

7.61 

10.75 

>.05 


>.05 


<.05* 


>.05 


>.05 


<.01* 


<01** 


<.05* 


Table  2  Continued 


;TAI  Trait 

M 

27.31 

33.33 

35.86 

SD 

7.78 

9.63 

13.82 

>.05 
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Table  2  Continued 

Note:  M=Mean;  SD==Standard  deviation;  NAART  FSIQ=North  American  Adult  Reading 
Test  Full  Scale  IQ;  PTA=Post-traumatic  amnesia;  BDI=Beck  Depression  Inventory; 
STAI=State  Trait  Anxiety  Inventory;  ^=Mean  rank  used  for  Mann- Whitney  U  statistic; 
*=Mod/Sev  TBI  significantly  different  fi-om  Mild  TBI;  **=Mod/Sev  TBI  significantly 
different  fi-om  Controls. 

TBI  participants  sustained  their  injuries  in  the  following  manner:  25  (54%)  were 
involved  in  motor  vehicle  vs.  motor  vehicle  accidents,  10  (22%)  were  involved  in 
sport/recreational  accidents,  6  (13%)  were  injured  in  motor  vehicle  vs.  pedestrian 
accidents,  and  5  (11%)  were  injured  in  falls.  Eight  (17%)  TBI  participants  reported 
having  been  in  litigation  related  to  their  accident  in  the  past,  one  participant  (2%)  planned 
litigation  in  the  fixture,  but  no  participant  was  actively  engaged  in  litigation  at  the  time  of 
testing. 

Hypothesis/Prediction  I 

Since  we  were  interested  in  determining  if  groups  that  differed  in  brain  injury 
severity  performed  differently  on  individual  tests  of  executive  fiinctioning/control,  we 
predicted  that  controls  would  perform  best  and  mild  TBI  participants  would  perform 
better  than  moderate/severe  TBI  participants.  Given  the  univariate  nature  of  our  interest 
(Huberty  &  Morris,  1989),  we  conducted  multiple  one-way  (3 -Group)  ANOVAs  to 
determine  if  between  group  differences  existed.  Eta^  was  used  as  a  measure  of  the 
proportion  of  variance  accounted  for  by  the  independent  variable  (Howell,  1992). 
Following  omnibus  ANOVAs,  we  followed-up  with  pair-wise  comparisons  to  determine 
which  group  differences  accounted  for  the  significant  overall  ANOVA.  P:ivalues  for  pair- 
wise  comparisons  are  provided  below  in  parentheses. 
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Cognitive  Psychological  Tests 

Results  indicated  that  moderate/severe  TBI  participants  made  a  significantly 
greater  proportion  of  errors  on  the  N-Back  (Low)  task  than  did  control  participants 
(p<.05;  see  Table  3).  On  the  N-Back  (High)  task,  moderate/severe  TBI  participants  made 
a  significantly  greater  proportion  or  errors  than  did  mild  TBI  (p<.01)  and  control 
participants  (p<.001).  On  the  computerized  Stroop  task,  moderate/severe  TBI 
participants  made  a  significantly  greater  proportion  of  errors  on  the  color-long 
incongruent  trials  than  did  mild  TBI  (p<.05)  and  control  participants  (p<.05);  however, 
no  between-group  differences  were  observed  on  the  color-short  incongruent  trials. 
Moderate/severe  TBI  participants  achieved  a  significantly  lower  d'  on  the  AX-Continuous 
Performance  Test  (CPT)  long  trials  than  did  mild  TBI  (p<.01)  and  control  participants 
(p<.01).  They  also  obtained  a  significantly  lower  d'  on  the  AX-CPT  Interference  long 
trials  than  did  mild  TBI  (p<.05)  and  control  participants  (p<.05).  Moderate/severe  TBI 
participants  completed  significantly  fewer  categories  on  the  computerized  Wisconsin  Card 
Sorting  Test  (WCST)  than  did  mild  TBI  (p<.01)  and  control  participants  (e<.01). 
Moderate/severe  TBI  individuals  made  a  significantly  greater  proportion  of  perseverative 
errors  on  the  computerized  WCST  than  did  mild  TBI  (p<.01)  and  control  participants 
(p<.001;  see  Figure  1  for  z-scores). 
Traditional  Neuropsychological  Tests 

Analyses  revealed  no  reliable  group  differences  in  the  number  of  categories 
completed  or  proportion  of  perseverative  ertors  on  the  card  version  of  the  WCST.  On  the 
traditional  card  version  of  the  Stroop  test,  moderate/severe  TBI  participants  achieved  an 
interference  score  significantly  lower  than  that  of  controls  (p<.01).  They  took 
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significantly  longer  to  complete  the  Trail  Making  Test  Part  A  when  compared  to  mild  TBI 
(p<.01)  and  control  (2<.001)  participants.  Moderate/severe  TBI  participants  required  a 
longer  period  of  time  to  complete  the  Trail  Making  Test  Part  B  when  compared  to  mild 
(p<.05)  and  control  (p<.001)  participants.  There  were  no  differences  between  groups  on 
the  Digit  Span  forward  or  backward  subtests.  On  the  PASAT,  moderate/severe  TBI 
participants  achieved  a  significantly  lower  overall  total  score  than  did  mild  TBI 
participants  (e<.05)  or  controls  (p<.001).  Moderate/severe  TBI  participants  completed 
significantly  fewer  items  on  the  Digit  Symbol  task  than  did  controls  (p<.001).  On  the 
modified  version  of  the  Neurobehavioral  Rating  Scale  -  Revised  (NRS-R), 
moderate/severe  TBI  participants  rated  themselves  as  experiencing  significantly  greater 
cognitive  and  behavioral  problems  than  did  mild  TBI  participants  (e<.05)  or  controls 
(p<.001).  Informants  of  moderate/severe  TBI  participants  rated  them  as  having 
significantly  greater  cognitive  and  behavioral  problems  than  did  the  informants  of  mild  TBI 
participants  (p<.05)  or  controls  (p<.01).  On  the  dysexecutive  questionnaire, 
moderate/severe  TBI  participants  rated  themselves  as  experiencing  significantly  greater 
executive  functioning  deficits  than  did  controls  (p<.01),  and  informants  of 
moderate/severe  TBI  participants  rated  them  as  having  greater  executive  fianctioning 
deficits  than  did  the  informants  of  controls  (p<.05;  see  Figure  2  for  z-scores). 
Tests  of  Everyday  Executive  Dysfunction 

On  the  Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  Rule  Shift 
Cards  subtest,  moderate/severe  TBI  participants  performed  significantly  worse  than 
controls  (p<.05).  No  significant  differences  between  groups  were  found  on  the  Action 
Profile  Score  (see  Table  3;  see  Figure  3  for  z-scores). 
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Figure  1.  Z-scores  for  groups  across  cognitive  psychological  tests. 


"Means  and  standard  deviations  are  based  on  controls  from  this  study.  Z-scores  that  were 
originally  positive  but  reflected  greater  impairment  (e.g.,  proportion  of  errors)  were 
changed  to  negative  z-scores  for  continuity  of  data  presentation. 
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Figure  2.  Z-scores  for  groups  across  traditional  neuropsychological  tests. 

^Means  and  standard  deviations  are  based  on  controls  from  this  study.  Z-scores  that  were 
originally  positive  but  reflected  greater  impairment  (e.g.,  proportion  of  errors)  were 
changed  to  negative  z-scores  for  continuity  of  data  presentation. 


47 


o 
o 

I 
N 


Mild  TBI 
Mod/Sev  TBI 


=:<>^ 


.^ 


.^^' 


P<> 


.'!^ 


^ 


e^ 


1? 


_^^^ 
^-^ 


)^ 


><> 


.# 


<<^ 


<> 


^ 


Figure  3 .  Z-scores  for  groups  across  tests  of  everyday  executive  dysfunction. 


"Means  and  standard  deviations  are  based  on  controls  from  this  study,  Z-scores  that  were 
originally  positive  but  reflected  greater  impairment  (e.g.,  proportion  of  errors)  were 
changed  to  negative  z-scores  for  continuity  of  data  presentation. 
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Hypothesis/Prediction  II 

Here,  our  intentions  were  to  define  the  operating  characteristics  of  each  test  and 
then  determine  if  positive  predictive  power  differed  as  a  function  of  test  type  (e.g., 
cognitive,  neuropsychological,  test  of  everyday  executive  dysfunction).  We  hypothesized 
that  cognitive  psychological  tests  would  yield  higher  PPP  because  they  are  thought  to  be 
'purer'  measures  of  executive  dysflinction  than  other  tests.  Specifically,  we  predicted  that 
cognitive  psychological  tests  of  executive  dysflinction  would  yield  greater  positive 
predictive  power  than  traditional  neuropsychological  tests  or  tests  of  everyday  executive 
dysfunction  (e.g.,  Behavioral  Assessment  of  the  Dysexecutive  Syndrome  subtests). 

Rather  than  the  non-uniform  method  of  using  cutoff  scores  as  indices  of  impaired 
test  performance,  all  raw  scores  were  converted  to  z-scores  based  on  TBI  and  control 
participant  data  from  the  current  sample.  This  method  was  preferred  over  the  cutoff  score 
approach  because  it  uses  local  control  data  that  are  likely  to  match  sample  TBI 
participants  with  respect  to  important  demographic  features  (e.g.,  geographic  region, 
quality  of  education).  In  addition,  the  local  control  method  permitted  us  to  use  the  same 
cutoff  threshold  (e.g.,  1 .0,  1 .5,  2.0  standard  deviations)  across  all  tests  as  opposed  to 
published  cutoff  scores  that  may  be  unique  to  each  test. 

Prevalence  rates  of  brain  injury  were  determined  for  each  test  by  dividing  the 
number  of  participants  with  brain  damage  present  by  the  total  sample  size.  Sensitivity  was 
computed  by  dividing  the  number  of  true  positives  by  the  total  number  of  participants  with 
brain  injury.  Specificity  was  computed  by  dividing  the  number  of  true  negatives  by  the 
total  number  of  participants  without  brain  injury.  Positive  predictive  power  (PPP)  was 
computed  by  dividing  the  number  of  true  positives  by  the  total  number  of  participants  with 
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a  positive  sign  (e.g.,  impaired  test  finding).  Negative  predictive  power  (NPP)  was 
computed  by  dividing  the  number  of  true  negatives  by  the  total  number  of  participants 
with  a  negative  sign  (e.g.,  non-impaired  test  finding).  Overall  predictive  power  (OPP) 
was  computed  by  adding  the  total  number  of  true  positives  and  true  negatives  and  dividing 
this  sum  by  the  total  sample  size.  Ihe  false  negative  rate  was  computed  by  subtracting 
the  negative  predictive  power  coefficient  fi"om  1.0.  Odds  ratios  (OR)  were  computed  by 
dividing  the  dependent  variable's  sensitivity  by  its  false  positive  rate.  Predictive  power 
estimates  were  interpreted  using  Cohen's  (1988)  assertion  that  coefficients  of 
approximately  .20,  ,50  and  .80  are  small,  medium  and  large,  respectively. 
Operating  Characteristics  I.  Control  (n=24)  and  TBI  (n=46)  participants 

The  results  fi-om  the  statistical  computations  comparing  test  diagnostic  accuracy 
between  controls  and  all  TBI  participants  are  presented  in  Table  4.  Given  the  large 
amount  of  data  obtained  in  the  current  study,  narrative  descriptions  will  be  provided  only 
for  the  1.0  standard  deviation  level  of  impairment  (e.g.,  16*  %tile)  for  each  dependent 
variable. 
Cognitive  psychological  tests 

N-Back  (Low).  This  variable  showed  an  OPP  of  56%,  and  sensitivity  and 
specificity  estimates  of  39%  and  88%,  respectively.  These  findings  indicate  that  61%  of 
individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  86% 
of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  43%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
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standard  deviation  or  more  below  the  mean  of  controls  is  3. 1  times  more  likely  to  have 
brain  dysfunction  than  a  person  who  scores  above  that  level. 

N-Back  (High).  This  variable  showed  an  OPP  of  57%,  and  sensitivity  and 
specificity  estimates  of  46%  and  79%,  respectively.  These  findings  indicate  that  54%  of 
individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  81% 
of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  43%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
standard  deviation  or  more  below  the  mean  of  controls  is  2.2  times  more  likely  to  have 
brain  dysfimction  than  a  person  who  scores  above  that  level. 

Stroop  (Color-Long  Incongruent).  This  variable  showed  an  OPP  of  5 1%,  and 
sensitivity  and  specificity  estimates  of  33%  and  88%,  respectively.  These  findings  indicate 
that  67%  of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  83%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated 
that  40%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative) 
and  were  correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an 
individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  2.6 
times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Stroop  (Color-Short  Incongruent).  This  variable  showed  an  OPP  of  44%,  and 
sensitivity  and  specificity  estimates  of  26%  and  79%,  respectively.  These  findings  indicate 
that  74%  of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate 
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indicated  that  71%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated 
that  36%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative) 
and  were  correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an 
individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  1.3 
times  more  likely  to  have  brain  dysfianction  than  a  person  who  scores  above  that  level. 

d'  AX  Noint  Long.  This  variable  showed  an  OPP  of  56%,  and  sensitivity  and 
specificity  estimates  of  37%  and  92%,  respectively.  These  findings  indicate  that  63%  of 
individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  90% 
of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  43%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
standard  deviation  or  more  below  the  mean  of  controls  is  4.4  times  more  likely  to  have 
brain  dysfijnction  than  a  person  who  scores  above  that  level. 

d'  AX  Int  Long.  This  variable  showed  an  OPP  of  50%,  and  sensitivity  and 
specificity  estimates  of  35%  and  79%,  respectively.  These  findings  indicate  that  65%  of 
individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  76% 
of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  39%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
standard  deviation  or  more  below  the  mean  of  controls  is  1 .7  times  more  likely  to  have 
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brain  dysfunction  than  a  person  who  scores  above  that  level. 

WCST  Computer-Cat.  Completed.  This  variable  showed  an  OPP  of  49%,  and 
sensitivity  and  specificity  estimates  of  24%  and  96%,  respectively.  These  findings  indicate 
that  76%  of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  92%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated 
that  40%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative) 
and  were  correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an 
individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  5.7 
times  more  likely  to  have  brain  dysfimction  than  a  person  who  scores  above  that  level. 

WCST  Computer-  Prop.  Persev.  Errors.  This  variable  showed  an  OPP  of  59%, 
and  sensitivity  and  specificity  estimates  of  39%  and  96%,  respectively.  These  findings 
indicate  that  61%  of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  95%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated 
that  45%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative) 
and  were  correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an 
individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  2.5 
times  more  likely  to  have  brain  dysfunction  than  a  person  who  scores  above  that  level. 
Traditional  neuropsychological  tests 

WCST  Card  -  Cat.  Completed.  This  variable  showed  an  OPP  of  38%,  and 
sensitivity  and  specificity  estimates  of  1 5%  and  91%,  respectively.  These  findings  indicate 
that  85%  of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate 
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indicated  that  80%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated 
that  31%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative) 
and  were  correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an 
individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  1 .7 
times  more  likely  to  have  brain  dysfianction  than  a  person  who  scores  above  that  level. 

WCST  Card  -  Prop.  Persev.  Errors.  This  variable  showed  an  OPP  of  43%,  and 
sensitivity  and  specificity  estimates  of  23%  and  91%,  respectively.  These  findings  indicate 
that  77%  of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  86%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated 
that  33%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative) 
and  were  correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an 
individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  2.5 
times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Stroop  -  Interference.  This  variable  showed  an  OPP  of  53%,  and  sensitivity  and 
specificity  estimates  of  37%  and  83%,  respectively.  These  findings  indicate  that  63%  of 
individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  81% 
of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  41%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
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standard  deviation  or  more  below  the  mean  of  controls  is  2.2  times  more  likely  to  have 
brain  dysfunction  than  a  person  who  scores  above  that  level. 

Trail  Making  Test  Part  A.  This  variable  showed  an  OPP  of  63%,  and  sensitivity 
and  specificity  estimates  of  52%  and  83%,  respectively.  These  findings  indicate  that  48% 
of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
86%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  48%  of 
control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who 
scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  3. 1  times  more  likely 
to  have  brain  dysfianction  than  a  person  who  scores  above  that  level. 

Trail  Making  Test  Part  B.  This  variable  showed  an  OPP  of  67%,  and  sensitivity 
and  specificity  estimates  of  59%  and  83%,  respectively.  These  findings  indicate  that  41% 
of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
87%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  51%  of 
control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who 
scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  3.5  times  more  likely 
to  have  brain  dysfianction  than  a  person  who  scores  above  that  level. 

Digit  Span  Forward.  This  variable  showed  an  OPP  of  47%,  and  sensitivity  and 
specificity  estimates  of  33%  and  75%,  respectively.  These  findings  indicate  that  67%  of 
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individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  71% 
of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  37%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
standard  deviation  or  more  below  the  mean  of  controls  is  1 .3  times  more  likely  to  have 
brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Digit  Span  Backward.  This  variable  showed  an  OPP  of  50%,  and  sensitivity  and 
specificity  estimates  of  26%  and  96%,  respectively.  These  findings  indicate  that  74%  of 
individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  92% 
of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  40%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
standard  deviation  or  more  below  the  mean  of  controls  is  6.3  times  more  likely  to  have 
brain  dysfiinction  than  a  person  who  scores  above  that  level. 

PAS  AT  -  Total.  This  variable  showed  an  OPP  of  60%,  and  sensitivity  and 
specificity  estimates  of  50%  and  79%,  respectively.  These  findings  indicate  that  50%  of 
individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  82% 
of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  45%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
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classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
standard  deviation  or  more  below  the  mean  of  controls  is  2.4  times  more  likely  to  have 
brain  dysfunction  than  a  person  who  scores  above  that  level. 

Digit  Svmbol.  This  variable  showed  an  OPP  of  62%,  and  sensitivity  and  specificity 
estimates  of  50%  and  83%,  respectively.  These  findings  indicate  that  50%  of  individuals 
with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  83%  of 
participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  50%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
standard  deviation  or  more  below  the  mean  of  controls  is  2.9  times  more  likely  to  have 
brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Modified  Neurobehavioral  Rating  Scale  -  Revised  -  Self  Report.  This  variable 
showed  an  OPP  of  87%,  and  sensitivity  and  specificity  estimates  of  57%  and  83%, 
respectively.  These  findings  indicate  that  43%  of  individuals  with  a  TBI  scored  in  the 
normal  range.  The  PPP  estimate  indicated  that  87%  of  participants  who  obtained  a  score 
in  the  impaired  range  (test  positive)  were  correctly  classified  as  having  sustained  a  TBI. 
The  NPP  estimate  indicated  that  50%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the 
mean  of  controls  is  3.4  times  more  likely  to  have  brain  dysfiinction  than  a  person  who 
scores  above  that  level. 
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Modified  Neurobehavioral  Rating  Scale  -  Revised  -  Informant  Report .  This 
variable  showed  an  OPP  of  63%,  and  sensitivity  and  specificity  estimates  of  49%  and 
90%,  respectively.  These  findings  indicate  that  51%  of  individuals  with  a  TBI  scored  in 
the  normal  range.  The  PPP  estimate  indicated  that  91%)  of  participants  who  obtained  a 
score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having  sustained  a 
TBI.  The  NPP  estimate  indicated  that  47%)  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the 
mean  of  controls  is  4.9  times  more  likely  to  have  brain  dysfiinction  than  a  person  who 
scores  above  that  level. 

Dysexecutive  Questionnaire  -  Self  Report.  This  variable  showed  an  OPP  of  60%), 
and  sensitivity  and  specificity  estimates  of  46%)  and  88%),  respectively.  These  findings 
indicate  that  54%)  of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  88%)  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated 
that  46%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative) 
and  were  correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an 
individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  3.7 
times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Dysexecutive  Questionnaire  -  Informant  Report.  This  variable  showed  an  QPP  of 
53%,  and  sensitivity  and  specificity  estimates  of  36%)  and  85%),  respectively.  These 
findings  indicate  that  64%)  of  individuals  with  a  TBI  scored  in  the  normal  range.  The  PPP 
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estimate  indicated  that  82%  of  participants  who  obtained  a  score  in  the  impaired  range 
(test  positive)  were  correctly  classified  as  having  sustained  a  TBI.  The  NPP  estimate 
indicated  that  41%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  TBI.  For  this  variable,  an 
individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  2.4 
times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 
Tests  of  everyday  executive  dysfunction 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Rule  Shift 
Cards.  This  variable  showed  an  OPP  of  56%,  and  sensitivity  and  specificity  estimates  of 
37%  and  92%,  respectively.  These  findings  indicate  that  63%  of  individuals  Avith  a  TBI 
scored  in  the  normal  range.  The  PPP  estimate  indicated  that  90%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  TBI.  The  NPP  estimate  indicated  that  43%  of  control  participants  achieved  a 
score  in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having 
sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation  or 
more  below  the  mean  of  controls  is  4.4  times  more  likely  to  have  brain  dysfijnction  than  a 
person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Action  Program 
Test.  This  variable  showed  an  OPP  of  44%,  and  sensitivity  and  specificity  estimates  of 
20%  and  92%,  respectively.  These  findings  indicate  that  80%  of  individuals  with  a  TBI 
scored  in  the  normal  range.  The  PPP  estimate  indicated  that  82%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
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sustained  a  TBI.  The  NPP  estimate  indicated  that  37%  of  control  participants  achieved  a 
score  in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having 
sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation  or 
more  below  the  mean  of  controls  is  2.4  times  more  likely  to  have  brain  dysfunction  than  a 
person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Key  Search 
Test.  This  variable  showed  an  OPP  of  56%,  and  sensitivity  and  specificity  estimates  of 
37%  and  92%,  respectively.  These  findings  indicate  that  65%  of  individuals  with  a  TBI 
scored  in  the  normal  range.  The  PPP  estimate  indicated  that  90%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  TBI.  The  NPP  estimate  indicated  that  43%  of  control  participants  achieved  a 
score  in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having 
sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation  or 
more  below  the  mean  of  controls  is  4.4  times  more  likely  to  have  brain  dysfianction  than  a 
person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Zoo  Map  Test. 
This  variable  showed  an  OPP  of  50%),  and  sensitivity  and  specificity  estimates  of  30%  and 
88%,  respectively.  These  findings  indicate  that  70%.  of  individuals  with  a  TBI  scored  in 
the  normal  range.  The  PPP  estimate  indicated  that  82%  of  participants  who  obtained  a 
score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having  sustained  a 
TBI.  The  NPP  estimate  indicated  that  40%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
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TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the 
mean  of  controls  is  2.4  times  more  likely  to  have  brain  dysfunction  than  a  person  who 
scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Modified  Six 
Elements  Test.  This  variable  showed  an  OPP  of  40%,  and  sensitivity  and  specificity 
estimates  of  15%  and  88%,  respectively.  These  findings  indicate  that  85%  of  individuals 
with  a  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  70%  of 
participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  TBI.  The  NPP  estimate  indicated  that  35%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one 
standard  deviation  or  more  below  the  mean  of  controls  is  1 .2  times  more  likely  to  have 
brain  dysfijnction  than  a  person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Total  Profile 
Score.  This  variable  showed  an  OPP  of  46%,  and  sensitivity  and  specificity  estimates  of 
26%  and  83%,  respectively.  These  findings  indicate  that  74%  of  individuals  with  a  TBI 
scored  in  the  normal  range.  The  PPP  estimate  indicated  that  75%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  TBI.  The  NPP  estimate  indicated  that  37%  of  control  participants  achieved  a 
score  in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having 
sustained  a  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation  or 
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more  below  the  mean  of  controls  is  1 .6  times  more  likely  to  have  brain  dysfunction  than  a 

person  who  scores  above  that  level. 

Operating  Characteristics  II:  Control  (n=24)  and  mild  TBI  (n=20)  participants. 

The  results  from  the  statistical  computations  comparing  test  diagnostic  accuracy 
between  controls  and  mild  TBI  participants  are  presented  in  Table  5.  Given  the  large 
amount  of  data  obtained  in  the  current  study,  narrative  descriptions  will  only  be  provided 
for  the  1.0  standard  deviation  level  of  impairment  (e.g.,  16""  %tile)  for  each  dependent 
variable. 
Cognitive  psychological  tests 

N-Back  (Low).  This  variable  showed  an  OPP  of  64%,  and  sensitivity  and 
specificity  estimates  of  35%  and  88%,  respectively.  These  findings  indicate  that  65%  of 
individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
70%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  62% 
of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  2.8  times  more 
likely  to  have  brain  dysfijnction  than  a  person  who  scores  above  that  level. 

N-Back  (High).  This  variable  showed  an  OPP  of  55%,  and  sensitivity  and 
specificity  estimates  of  25%  and  79%,  respectively.  These  findings  indicate  that  75%  of 
individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
50%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
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correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  56% 
of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  1.2  times  more 
likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Stroop  (Color-Long  Incongruent).  This  variable  showed  an  OPP  of  55%,  and 
sensitivity  and  specificity  estimates  of  15%  and  88%,  respectively.  These  findings  indicate 
that  85%  of  individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  50%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate 
indicated  that  55%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this 
variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of 
controls  is  1 .2  times  more  likely  to  have  brain  dysfianction  than  a  person  who  scores  above 
that  level. 

Stroop  (Color-Short  Incongruent).  This  variable  showed  an  OPP  of  52%,  and 
sensitivity  and  specificity  estimates  of  20%  and  79%,  respectively.  These  findings  indicate 
that  80%  of  individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  44%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate 
indicated  that  54%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this 
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variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of 
controls  is  .96  times  more  likely  to  have  brain  dysfunction  than  a  person  who  scores  above 
that  level. 

d'  AX  Noint  Long.  This  variable  showed  an  OPP  of  59%,  and  sensitivity  and 
specificity  estimates  of  20%  and  92%,  respectively.  These  findings  indicate  that  80%  of 
individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
67%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  58% 
of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  2.4  times  more 
likely  to  have  brain  dysfianction  than  a  person  who  scores  above  that  level. 

d'  AX  Int  Long.  This  variable  showed  an  OPP  of  52%,  and  sensitivity  and 
specificity  estimates  of  20%  and  79%,  respectively.  These  findings  indicate  that  80%  of 
individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
44%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  54% 
of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  .96  times  more 
likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 
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WCST  Computer-Cat.  Completed.  This  variable  showed  an  OPP  of  5 5%,  and 
sensitivity  and  specificity  estimates  of  5%  and  96%,  respectively.  These  findings  indicate 
that  95%  of  individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  50%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate 
indicated  that  55%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this 
variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of 
controls  is  1 .2  times  more  likely  to  have  brain  dysfijnction  than  a  person  who  scores  above 
that  level. 

WCST  Computer-  Prop.  Persev.  Errors.  This  variable  showed  an  OPP  of  61%, 
and  sensitivity  and  specificity  estimates  of  20%  and  96%,  respectively.  These  findings 
indicate  that  80%  of  individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP 
estimate  indicated  that  80%  of  participants  who  obtained  a  score  in  the  impaired  range 
(test  positive)  were  correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate 
indicated  that  59%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this 
variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of 
controls  is  4.8  times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above 
that  level. 
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Traditional  neuropsychological  tests 

WCST  Card  -  Cat.  Completed.  This  variable  showed  an  OPP  of  42%,  and 
sensitivity  and  specificity  estimates  of  7%  and  91%,  respectively.  These  findings  indicate 
that  93%  of  individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  50%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate 
indicated  that  42%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this 
variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of 
controls  is  .73  times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above 
that  level. 

WCST  Card  -  Prop.  Persev.  Errors.  This  variable  showed  an  OPP  of  50%,  and 
sensitivity  and  specificity  estimates  of  20%  and  91%,  respectively.  These  findings  indicate 
that  80%  of  individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  75%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate 
indicated  that  45%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  mild  TBI,  For  this 
variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of 
controls  is  2.2  times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above 
that  level. 
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Stroop  -  Interference.  This  variable  showed  an  OPP  of  57%,  and  sensitivity  and 
specificity  estimates  of  25%  and  83%,  respectively.  These  findings  indicate  that  75%  of 
individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
56%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  57% 
of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  1.5  times  more 
likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Trail  Making  Test  Part  A.  This  variable  showed  an  OPP  of  64%,  and  sensitivity 
and  specificity  estimates  of  40%  and  83%,  respectively.  These  findings  indicate  that  60% 
of  individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated 
that  67%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  63% 
of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  2.4  times  more 
likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Trail  Making  Test  Part  B.  This  variable  showed  an  OPP  of  64%,  and  sensitivity 
and  specificity  estimates  of  40%  and  83%,  respectively.  These  findings  indicate  that  60% 
of  individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated 
that  67%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  63% 
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of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  2.4  times  more 
likely  to  have  brain  dysfunction  than  a  person  who  scores  above  that  level. 

Digit  Span  Forward.  This  variable  showed  an  OPP  of  52%,  and  sensitivity  and 
specificity  estimates  of  25%  and  75%,  respectively.  These  findings  indicate  that  75%  of 
individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
46%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  55% 
of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  1 .0  time  more 
likely  to  have  brain  dysfianction  than  a  person  who  scores  above  that  level. 

Digit  Span  Backward    This  variable  showed  an  OPP  of  59%,  and  sensitivity  and 
specificity  estimates  of  1 5%  and  96%,  respectively.  These  findings  indicate  that  85%  of 
individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
75%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  58% 
of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  3.6  times  more 
likely  to  have  brain  dysfijnction  than  a  person  who  scores  above  that  level. 
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PASAT  -  Total.  This  variable  showed  an  OPP  of  57%,  and  sensitivity  and 
specificity  estimates  of  30%  and  79%,  respectively.  These  findings  indicate  that  70%  of 
individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
55%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  58% 
of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were 
correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  1 .4  times  more 
likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Digit  Symbol.  This  variable  showed  an  OPP  of  65%,  and  sensitivity  and  specificity 
estimates  of  45%  and  83%,  respectively.  These  findings  indicate  that  55%  of  individuals 
with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  69%  of 
participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  63%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual  who  scores 
one  standard  deviation  or  more  below  the  mean  of  controls  is  2.6  times  more  likely  to 
have  brain  dysfijnction  than  a  person  who  scores  above  that  level. 

Modified  Neurobehavioral  Rating  Scale  -  Revised  -  Self  Report.  This  variable 
showed  an  OPP  of  64%,  and  sensitivity  and  specificity  estimates  of  40%  and  83%, 
respectively.  These  findings  indicate  that  60%  of  individuals  with  a  mild  TBI  scored  in  the 
normal  range.  The  PPP  estimate  indicated  that  67%  of  participants  who  obtained  a  score 
in  the  impaired  range  (test  positive)  were  correctly  classified  as  having  sustained  a  mild 
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TBI.  The  NPP  estimate  indicated  that  63%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
mild  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation  or  more 
below  the  mean  of  controls  is  2.4  times  more  likely  to  have  brain  dysfunction  than  a 
person  who  scores  above  that  level. 

Modified  Neurobehavioral  Rating  Scale  -  Revised  -  Significant  Other  Report . 
This  variable  showed  an  OPP  of  66%,  and  sensitivity  and  specificity  estimates  of  33%  and 
90%,  respectively.  These  findings  indicate  that  67%  of  individuals  with  a  mild  TBI  scored 
in  the  normal  range.  The  PPP  estimate  indicated  that  71%  of  participants  who  obtained  a 
score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having  sustained  a 
mild  TBI.  The  NPP  estimate  indicated  that  64%  of  control  participants  achieved  a  score 
in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having 
sustained  a  mild  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  3.3  times  more  likely  to  have  brain  dysfunction  than 
a  person  who  scores  above  that  level. 

Dysexecutive  Questionnaire  -  Self  Report.  This  variable  showed  an  OPP  of  64%, 
and  sensitivity  and  specificity  estimates  of  35%  and  88%,  respectively.  These  findings 
indicate  that  65%  of  individuals  with  a  mild  TBI  scored  in  the  normal  range.  The  PPP 
estimate  indicated  that  70%  of  participants  who  obtained  a  score  in  the  impaired  range 
(test  positive)  were  correctly  classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate 
indicated  that  62%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  mild  TBI.  For  this 
variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of 
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controls  is  1.8  times  more  likely  to  have  brain  dysfunction  than  a  person  who  scores  above 
that  level. 

Dysexecutive  Questionnaire  -  Significant  Other  Report.  This  variable  showed  an 
OPP  of  60%,  and  sensitivity  and  specificity  estimates  of  27%  and  85%,  respectively. 
These  findings  indicate  that  73%  of  individuals  with  a  mild  TBI  scored  in  the  normal 
range.  The  PPP  estimate  indicated  that  57%  of  participants  who  obtained  a  score  in  the 
impaired  range  (test  positive)  were  correctly  classified  as  having  sustained  a  mild  TBI. 
The  NPP  estimate  indicated  that  61%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
mild  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation  or  more 
below  the  mean  of  controls  is  1.8  times  more  likely  to  have  brain  dysfianction  than  a 
person  who  scores  above  that  level. 
Tests  of  everyday  executive  dysfijnction 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Rule  Shift 
Cards.  This  variable  showed  an  OPP  of  64%,  and  sensitivity  and  specificity  estimates  of 
30%  and  92%,  respectively.  These  findings  indicate  that  70%  of  individuals  with  a  mild 
TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  75%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  61%  of  control  participants 
achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as 
not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual  who  scores  one  standard 
deviation  or  more  below  the  mean  of  controls  is  3.6  times  more  likely  to  have  brain 
dysfijnction  than  a  person  who  scores  above  that  level. 
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Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Action  Program 
Test.  This  variable  showed  an  OPP  of  55%,  and  sensitivity  and  specificity  estimates  of 
10%  and  92%,  respectively.  These  findings  indicate  that  90%  of  individuals  with  a  mild 
TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  50%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  55%  of  control  participants 
achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as 
not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual  who  scores  one  standard 
deviation  or  more  below  the  mean  of  controls  is  1 .2  times  more  likely  to  have  brain 
dysfiinction  than  a  person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Key  Search 
Test.  This  variable  showed  an  OPP  of  66%,  and  sensitivity  and  specificity  estimates  of 
35%  and  92%,  respectively.  These  findings  indicate  that  65%  of  individuals  with  a  mild 
TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  78%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  63%  of  control  participants 
achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as 
not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual  who  scores  one  standard 
deviation  or  more  below  the  mean  of  controls  is  4.2  times  more  likely  to  have  brain 
dysfijnction  than  a  person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Zoo  Map  Test. 
This  variable  showed  an  OPP  of  59%,  and  sensitivity  and  specificity  estimates  of  25%  and 
88%,  respectively.  These  findings  indicate  that  75%  of  individuals  with  a  mild  TBI  scored 
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in  the  normal  range.  The  PPP  estimate  indicated  that  63%  of  participants  who  obtained  a 
score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having  sustained  a 
mild  TBI.  The  NPP  estimate  indicated  that  58%  of  control  participants  achieved  a  score 
in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having 
sustained  a  mild  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  2.0  times  more  likely  to  have  brain  dysfianction  than 
a  person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Modified  Six 
Elements  Test.  This  variable  showed  an  OPP  of  57%,  and  sensitivity  and  specificity 
estimates  of  20%)  and  88%),  respectively.  These  findings  indicate  that  80%)  of  individuals 
with  a  mild  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  57%)  of 
participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  57%»  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual  who  scores 
one  standard  deviation  or  more  below  the  mean  of  controls  is  1 .6  times  more  likely  to 
have  brain  dysfianction  than  a  person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Total  Profile 
Score.  This  variable  showed  an  OPP  of  57%),  and  sensitivity  and  specificity  estimates  of 
25%)  and  83%o,  respectively.  These  findings  indicate  that  75%o  of  individuals  with  a  mild 
TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  56%)  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  mild  TBI.  The  NPP  estimate  indicated  that  57%)  of  control  participants 
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achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as 
not  having  sustained  a  mild  TBI.  For  this  variable,  an  individual  who  scores  one  standard 
deviation  or  more  below  the  mean  of  controls  is  1 .5  times  more  likely  to  have  brain 
dysfunction  than  a  person  who  scores  above  that  level. 
Operating  Characteristics  III:  Control  (n=24)  and  moderate/severe  TBI  (n=26) 
participants. 

The  results  fi-om  the  statistical  computations  comparing  the  diagnostic  accuracy 
between  controls  and  moderate/severe  TBI  participants  are  presented  in  Table  6.  Again, 
given  the  large  amount  of  data  obtained  in  the  current  study,  narrative  descriptions  will 
only  be  provided  for  the  1 .0  standard  deviation  level  of  impairment  (e.g.,  16*  %tile)  for 
each  dependent  variable. 
Cognitive  psychological  tests 

N-Back  (Low).  This  variable  showed  an  OPP  of  64%,  and  sensitivity  and 
specificity  estimates  of  42%  and  88%,  respectively.  These  findings  indicate  that  58%  of 
individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  79%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  58%  of  control  participants  achieved  a  score  in  the  unimpaired 
range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  3.4  times  more  likely  to  have  brain  dysfiinction  than 
a  person  who  scores  above  that  level. 
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N-Back  (High).  This  variable  showed  an  OPP  of  70%,  and  sensitivity  and 
specificity  estimates  of  62%  and  79%,  respectively.  These  findings  indicate  that  38%  of 
individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  76%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  66%  of  control  participants  achieved  a  score  in  the  unimpaired 
range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  3.0  times  more  likely  to  have  brain  dysfiinction  than 
a  person  who  scores  above  that  level. 

Stroop  (Color-Long  Incongruent).  This  variable  showed  an  OPP  of  66%,  and 
sensitivity  and  specificity  estimates  of  46%  and  88%,  respectively.  These  findings  indicate 
that  54%  of  individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP 
estimate  indicated  that  S0%  of  participants  who  obtained  a  score  in  the  impaired  range 
(test  positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The 
NPP  estimate  indicated  that  60%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  3.7  times  more  likely  to  have  brain  dysfianction  than 
a  person  who  scores  above  that  level. 

Stroop  (Color-Short  Incongruent).  This  variable  showed  an  OPP  of  54%,  and 
sensitivity  and  specificity  estimates  of  3 1%  and  79%,  respectively.  These  findings  indicate 
that  69%  of  individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP 


85 
estimate  indicated  that  62%  of  participants  who  obtained  a  score  in  the  impaired  range 
(test  positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The 
NPP  estimate  indicated  that  51%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  1 .5  times  more  likely  to  have  brain  dysfiinction  than 
a  person  who  scores  above  that  level. 

d'  AX  Noint  Long.  This  variable  showed  an  OPP  of  70%,  and  sensitivity  and 
specificity  estimates  of  50%  and  92%.,  respectively.  These  findings  indicate  that  50%  of 
individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  87%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  63%  of  control  participants  achieved  a  score  in  the  unimpaired 
range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  6.0  times  more  likely  to  have  brain  dysfijnction  than 
a  person  who  scores  above  that  level. 

d'  AX  Int  Long.  This  variable  showed  an  OPP  of  62%o,  and  sensitivity  and 
specificity  estimates  of  46%  and  79%,  respectively.  These  findings  indicate  that  54%  of 
individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  71%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  58%  of  control  participants  achieved  a  score  in  the  unimpaired 
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range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  2.2  times  more  likely  to  have  brain  dysfimction  than 
a  person  who  scores  above  that  level. 

WCST  Computer-Cat.  Completed.  This  variable  showed  an  OPP  of  66%,  and 
sensitivity  and  specificity  estimates  of  38%  and  96%,  respectively.  These  findings  indicate 
that  62%  of  individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP 
estimate  indicated  that  91%  of  participants  who  obtained  a  score  in  the  impaired  range 
(test  positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The 
NPP  estimate  indicated  that  59%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  9.2  times  more  likely  to  have  brain  dysfiinction  than 
a  person  who  scores  above  that  level. 

WCST  Computer-  Prop.  Persev.  Errors.  This  variable  showed  an  OPP  of  74%, 
and  sensitivity  and  specificity  estimates  of  54%  and  96%,  respectively.  These  findings 
indicate  that  46%)  of  individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range. 
The  PPP  estimate  indicated  that  93%  of  participants  who  obtained  a  score  in  the  impaired 
range  (test  positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI. 
The  NPP  estimate  indicated  that  66%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
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or  more  below  the  mean  of  controls  is  13.0  times  more  likely  to  have  brain  dysfunction 
than  a  person  who  scores  above  that  level. 
Traditional  neuropsychological  tests 

WCST  Card  -  Cat.  Completed.  This  variable  showed  an  OPP  of  59%,  and 
sensitivity  and  specificity  estimates  of  27%  and  91%,  respectively.  These  findings  indicate 
that  73%  of  individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP 
estimate  indicated  that  75%  of  participants  who  obtained  a  score  in  the  impaired  range 
(test  positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The 
NPP  estimate  indicated  that  56%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  3.0  times  more  likely  to  have  brain  dysfunction  than 
a  person  who  scores  above  that  level. 

WCST  Card  -  Prop.  Persev.  Errors.  This  variable  showed  an  OPP  of  59%,  and 
sensitivity  and  specificity  estimates  of  27%  and  91%,  respectively.  These  findings  indicate 
that  73%  of  individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP 
estimate  indicated  that  75%  of  participants  who  obtained  a  score  in  the  impaired  range 
(test  positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The 
NPP  estimate  indicated  that  56%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  3.0  times  more  likely  to  have  brain  dysfunction  than 
a  person  who  scores  above  that  level. 
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Stroop  -  Interference.  This  variable  showed  an  OPP  of  64%,  and  sensitivity  and 
specificity  estimates  of  46%  and  83%,  respectively.  These  findings  indicate  that  54%  of 
individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  75%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  59%  of  control  participants  achieved  a  score  in  the  unimpaired 
range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  2.8  times  more  likely  to  have  brain  dysfiinction  than 
a  person  who  scores  above  that  level. 

Trail  Making  Test  Part  A.  This  variable  showed  an  OPP  of  72%,  and  sensitivity 
and  specificity  estimates  of  62%  and  83%,  respectively.  These  findings  indicate  that  38% 
of  individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  80%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  67%  of  control  participants  achieved  a  score  in  the  unimpaired 
range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  3.7  times  more  likely  to  have  brain  dysfunction  than 
a  person  who  scores  above  that  level. 

Trail  Making  Test  Part  B.  This  variable  showed  an  OPP  of  78%,  and  sensitivity 
and  specificity  estimates  of  73%  and  83%,  respectively.  These  findings  indicate  that  27% 
of  individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
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indicated  that  83%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  74%  of  control  participants  achieved  a  score  in  the  unimpaired 
range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  4.4  times  more  likely  to  have  brain  dysfunction  than 
a  person  who  scores  above  that  level. 

Digit  Span  Forward.  This  variable  showed  an  OPP  of  56%,  and  sensitivity  and 
specificity  estimates  of  38%  and  75%,  respectively.  These  findings  indicate  that  62%  of 
individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  63%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  53%  of  control  participants  achieved  a  score  in  the  unimpaired 
range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  1 .5  times  more  likely  to  have  brain  dysfunction  than 
a  person  who  scores  above  that  level. 

Digit  Span  Backward.  This  variable  showed  an  OPP  of  64%,  and  sensitivity  and 
specificity  estimates  of  35%  and  96%,  respectively.  These  findings  indicate  that  65%  of 
individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  90%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  58%  of  control  participants  achieved  a  score  in  the  unimpaired 
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range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  8.3  times  more  likely  to  have  brain  dysfunction  than 
a  person  who  scores  above  that  level. 

PASAT  -  Total.  This  variable  showed  an  OPP  of  72%,  and  sensitivity  and 
specificity  estimates  of  65%  and  79%,  respectively.  These  findings  indicate  that  35%  of 
individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate 
indicated  that  77%  of  participants  who  obtained  a  score  in  the  impaired  range  (test 
positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP 
estimate  indicated  that  68%  of  control  participants  achieved  a  score  in  the  unimpaired 
range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  3. 1  times  more  likely  to  have  brain  dysfiinction  than 
a  person  who  scores  above  that  level. 

Digit  Symbol.  This  variable  showed  an  OPP  of  71%,  and  sensitivity  and  specificity 
estimates  of  56%  and  83%,  respectively.  These  findings  indicate  that  44%  of  individuals 
with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
71%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP  estimate 
indicated  that  70%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  moderate/severe  TBI. 
For  this  variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the  mean 
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of  controls  is  3.2  times  more  likely  to  have  brain  dysfunction  than  a  person  who  scores 
above  that  level. 

Modified  Neurobehavioral  Rating  Scale  -  Revised  -  Self  Report.  This  variable 
showed  an  OPP  of  76%,  and  sensitivity  and  specificity  estimates  of  69%  and  83%, 
respectively.  These  findings  indicate  that  3 1%  of  individuals  with  a  moderate/severe  TBI 
scored  in  the  normal  range.  The  PPP  estimate  indicated  that  82%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  moderate/severe  TBI.  The  NPP  estimate  indicated  that  71%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  moderate/severe  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  4.2  times  more 
likely  to  have  brain  dysfunction  than  a  person  who  scores  above  that  level. 

Modified  Neurobehavioral  Rating  Scale  -  Revised  -  Significant  Other  Report  . 
This  variable  showed  an  OPP  of  73%,  and  sensitivity  and  specificity  estimates  of  58%  and 
90%,  respectively.  These  findings  indicate  that  42%  of  individuals  with  a  moderate/severe 
TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  88%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  moderate/severe  TBI.  The  NPP  estimate  indicated  that  64%)  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  moderate/severe  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  5.8  times  more 
likely  to  have  brain  dysfunction  than  a  person  who  scores  above  that  level. 
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Dysexecutive  Questionnaire  -  Self  Report.  This  variable  showed  an  OPP  of  70%, 
and  sensitivity  and  specificity  estimates  of  54%  and  88%,  respectively.  These  findings 
indicate  that  46%  of  individuals  with  a  moderate/severe  TBI  scored  in  the  normal  range. 
The  PPP  estimate  indicated  that  82%  of  participants  who  obtained  a  score  in  the  impaired 
range  (test  positive)  were  correctly  classified  as  having  sustained  a  moderate/severe  TBI. 
The  NPP  estimate  indicated  that  64%  of  control  participants  achieved  a  score  in  the 
unimpaired  range  (test  negative)  and  were  correctly  classified  as  not  having  sustained  a 
moderate/severe  TBI.  For  this  variable,  an  individual  who  scores  one  standard  deviation 
or  more  below  the  mean  of  controls  is  4.3  times  more  likely  to  have  brain  dysfunction  than 
a  person  who  scores  above  that  level. 

Dysexecutive  Questionnaire  -  Significant  Other  Report.  This  variable  showed  an 
OPP  of  61%,  and  sensitivity  and  specificity  estimates  of  42%  and  85%,  respectively. 
These  findings  indicate  that  58%  of  individuals  with  a  moderate/severe  TBI  scored  in  the 
normal  range.  The  PPP  estimate  indicated  that  77%  of  participants  who  obtained  a  score 
in  the  impaired  range  (test  positive)  were  correctly  classified  as  having  sustained  a 
moderate/severe  TBI.  The  NPP  estimate  indicated  that  55%  of  control  participants 
achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly  classified  as 
not  having  sustained  a  moderate/severe  TBI.  For  this  variable,  an  individual  who  scores 
one  standard  deviation  or  more  below  the  mean  of  controls  is  2.8  times  more  likely  to 
have  brain  dysfimction  than  a  person  who  scores  above  that  level. 
Tests  of  everyday  executive  dysfijnction 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Rule  Shift 
Cards.  This  variable  showed  an  OPP  of  66%,  and  sensitivity  and  specificity  estimates  of 
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42%  and  92%,  respectively.  These  findings  indicate  that  58%  of  individuals  with  a 
moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  85%  of 
participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP  estimate  indicated  that 
59%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and 
were  correctly  classified  as  not  having  sustained  a  moderate/severe  TBI.  For  this  variable, 
an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  5.1 
times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  E)vsexecutive  Syndrome  (BADS)  -  Action  Program 
Test    This  variable  showed  an  OPP  of  58%,  and  sensitivity  and  specificity  estimates  of 
27%  and  92%,  respectively.  These  findings  indicate  that  73%  of  individuals  with  a 
moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  78%  of 
participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP  estimate  indicated  that 
54%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and 
were  correctly  classified  as  not  having  sustained  a  moderate/severe  TBI.  For  this  variable, 
an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  3.2 
times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Key  Search 
Test.  This  variable  showed  an  OPP  of  64%,  and  sensitivity  and  specificity  estimates  of 
38%  and  92%,  respectively.  These  findings  indicate  that  62%  of  individuals  with  a 
moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  83%  of 
participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
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classified  as  having  sustained  a  moderate/severe  TBI    The  NPP  estimate  indicated  that 
58%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and 
were  correctly  classified  as  not  having  sustained  a  moderate/severe  TBI.  For  this  variable, 
an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  4.6 
times  more  likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Zoo  Map  Test. 
This  variable  showed  an  OPP  of  60%,  and  sensitivity  and  specificity  estimates  of  35%  and 
88%,  respectively.  These  findings  indicate  that  65%  of  individuals  with  a  moderate/severe 
TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  75%  of  participants  who 
obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly  classified  as  having 
sustained  a  moderate/severe  TBI.  The  NPP  estimate  indicated  that  55%  of  control 
participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and  were  correctly 
classified  as  not  having  sustained  a  moderate/severe  TBI.  For  this  variable,  an  individual 
who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  2.8  times  more 
likely  to  have  brain  dysfiinction  than  a  person  who  scores  above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Syndrome  (BADS)  -  Modified  Six 
Elements  Test.  This  variable  showed  an  OPP  of  48%,  and  sensitivity  and  specificity 
estimates  of  12%  and  88%,  respectively.  These  findings  indicate  that  88%  of  individuals 
with  a  moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that 
50%  of  participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were 
correctly  classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP  estimate 
indicated  that  48%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test 
negative)  and  were  correctly  classified  as  not  having  sustained  a  moderate/severe  TBI. 
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For  this  variable,  an  individual  who  scores  one  standard  deviation  or  more  below  the  mean 
of  controls  is  .92  times  more  likely  to  have  brain  dysfunction  than  a  person  who  scores 
above  that  level. 

Behavioral  Assessment  of  the  Dysexecutive  Svndrome  (BADS)  -  Total  Profile 
Score.  This  variable  showed  an  OPP  of  54%,  and  sensitivity  and  specificity  estimates  of 
27%  and  83%,  respectively.  These  findings  indicate  that  73%  of  individuals  with  a 
moderate/severe  TBI  scored  in  the  normal  range.  The  PPP  estimate  indicated  that  64%  of 
participants  who  obtained  a  score  in  the  impaired  range  (test  positive)  were  correctly 
classified  as  having  sustained  a  moderate/severe  TBI.  The  NPP  estimate  indicated  that 
51%  of  control  participants  achieved  a  score  in  the  unimpaired  range  (test  negative)  and 
were  correctly  classified  as  not  having  sustained  a  moderate/severe  TBI.  For  this  variable, 
an  individual  who  scores  one  standard  deviation  or  more  below  the  mean  of  controls  is  1.6 
times  more  likely  to  have  brain  dysfijnction  than  a  person  who  scores  above  that  level. 

We  used  omnibus  one-way  ANOVAs  to  determine  if  cognitive  psychological  tests 
yielded  higher  positive  predictive  power  than  traditional  neuropsychological  tests  or  tests 
of  everyday  executive  dysfiinction.  Results  indicated  no  significant  differences  in  positive 
predictive  power  for  type  of  test  in  cases  of  mild  (p>.05)  and  moderate/severe  TBI 
(p>.05). 

Hypothesis/Prediction  III 

We  hypothesized  that  a  weighted  linear  combination  of  predictor  variables  would 
be  useful  in  differentiating  groups.  Specifically,  we  predicted  that  a  weighted  linear 
combination  of  predictor  variables  that  could  discriminate  group  membership  would 
include  a  greater  number  of  cognitive  psychological  tests  than  traditional 
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neuropsychological  tests  or  tests  of  everyday  executive  dysfunction.  We  performed  a  3- 
group  stepwise  predictive  discriminant  function  analysis  using  the  two  tests  with  the 
highest  positive  predictive  power  and  odds  ratio  from  cognitive  (e.g.,  computerized 
WCST  Proportion  of  Perseverative  Errors,  and  computerized  WCST  categories 
completed),  traditional  neuropsychological  (e.g..  Digit  Span  Backwards,  and  Dysexecutive 
Questionnaire  Self  Report),  and  everyday  executive  dysfunction  (e.g.,  BADS  Rule  Shifl 
Cards  and  Key  Search  Task)  tests  to  develop  a  weighted  linear  combination  of  predictor 
variables  that  would  best  discriminate  group  membership.  The  variables  used  were 
selected  from  the  1.0  level  of  impairment  for  the  entire  sample  (N  =  70). 

Of  the  original  70  cases,  six  were  identified  as  multivariate  outliers  - 
discontinuously  performing  more  than  three  standard  deviations  from  the  respective  mean 
of  the  entire  sample  on  one  or  more  variables.  All  six  of  these  cases  were  from  the 
moderate/severe  TBI  group,  and  they  were  dropped  from  the  analysis.  The  initial 
discriminant  function  was  performed  on  the  remaining  64  cases  (24  controls,  20  mild  TBI 
and  20  moderate/severe  TBI).  Unfortunately,  a  significant  Box's  M  test  (p  <  .001) 
indicated  heterogeneous  variance-covariance  matrices.  Tabachnick  and  Fidell  (2001) 
describe  the  Box's  M  test  as  "a  notoriously  sensitive  test"  (p.  330)  and  indicate  that  Box's 
M  test  can  be  disregarded  if  sample  sizes  are  equal  because  the  significance  test  is  then 
assumed  to  be  robust;  however,  if  sample  sizes  are  unequal,  robustness  is  not  guaranteed. 
With  this  in  mind,  four  control  cases  were  randomly  selected  and  dropped  from  the 
analysis  and  the  final  discriminant  function  was  performed  on  60  cases  (20  controls,  20 
mild  TBI  and  20  moderate/severe  TBI). 
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One  discriminant  function  resulted,  with  a  X^  (2)  =  9.00,  g  ==  .01 .  The  single 
discriminant  function  accounted  for  100%  of  the  between  group  variability,  and  table  7 
presents  the  standardized  discriminant  function  coefficients  and  structure  coefficients. 
Stevens  (1996)  recommends  using  the  standardized  discriminant  function  coefficients  to 
determine  the  contribution  (e.g.,  importance)  of  each  variable  to  the  discriminant  function. 
The  structure  coefficients  are  used  to  assign  a  meaningful  label  to  the  discriminant 
function. 


Table  7 

Standardized  Discriminant  Function  and  Structure  Coefficients 


Standardized 

discriminant 

function 

Structure 

Variable 

coefficients 

coefficients 

WCST  Prop.  Persev.  Errors 

1.00 

1.0 

WCST  Categories  Completed 

-.64 

DEX  Self 

.20 

Digit  Span  Backward 

-.17 

BADS  Key  Search 

-.06 

BADS  Rule  Shift  Cards 

.03 

Examination  of  the  standardized  discriminant  function  coefficients  indicated  that 
the  computerized  WCST  Proportion  of  Perseverative  errors  score  was  the  most  important 
predictor  variable  in  differentiating  groups.  As  seen  in  figure  4,  the  centroids  for  the 
control  and  mild  TBI  groups  were  located  on  the  negative  end  of  the  discriminant 
dimension,  while  the  centroid  of  the  moderate/severe  TBI  group  was  located  on  the 
positive  end  of  the  dimension.  The  positive  structure  coefficient  for  WCST  Proportion  of 
Perseverative  errors  indicates  that  this  variable  is  more  descriptive  of  groups  on  the 
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positive  end  of  the  dimension.  Based  on  past  neuropsychological  and  cognitive 
neuroscience  research  that  has  validated  the  use  of  the  WCST  in  various  clinical 
populations  (Heaton  et  al.,  1993;  Kimberg,  D'Esposito,  &  Farah,  1997;  Demakis,  2003), 
this  dimension  appears  to  represent  a  psychological  factor  associated  with  perseverative 
response  tendency. 


•     Controls 
■      Milds 
A     Mod/Sev 

• 

▲ 
■ 

1 

Discriminant  Function 


Figure  4.  Group  centroids  for  discriminant  function. 


DISCUSSION 
Between  Group  Performance  Differences 

One  of  the  primary  purposes  of  the  present  study  was  to  determine  if  controls, 
mild,  and  moderate/severe  TBI  participants  performed  differently  on  a  battery  of  executive 
fianctioning  tests.  We  hypothesized  that  group  mean  performance  scores,  on  individual 
tests  designed  to  measure  executive  dysfunction,  would  differ  as  a  function  of  group 
membership,  and  specifically  predicted  that  controls  would  have  the  best  performance 
followed  by  mild  TBI  and  then  moderate/severe  TBI  participants. 

Results  from  the  study  generally  supported  our  predictions.  On  18/27  dependent 
variables  controls  performed  best  followed  by  mild  TBI  and  then  moderate/severe  TBI 
participants.  No  statistically  significant  differences  between  controls  and  mild  TBI 
participants  were  found  on  any  dependent  variable  in  this  study;  however,  significant 
differences  between  controls  and  moderate/severe  TBI  participants  were  found  on  1 7/27 
dependent  variables.  Moderate/severe  TBI  participants  performed  significantly  poorer 
than  mild  TBI  participants  on  1 1/27  dependent  variables.  There  were  significant 
differences  between  groups  on  88%  of  cognitive  psychological  tests,  69%  of 
neuropsychological  tests  and  17%  of  tests  of  everyday  executive  dysfunction. 

An  interesting  finding  in  this  study  was  that  both  mild  and  moderate/severe  TBI 
patients  showed  greater  interference  on  the  card  Stroop  task  than  controls  (see  Table  3). 
This  finding  is  in  opposition  with  previous  findings  in  the  literature  (Rojas  &  Bennett, 
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1995;  Stuss  et  al.,  1985).  Most  of  these  studies  have  found  that  TBI  participants  show 
greater  interference  on  the  card  Stroop  based  on  group-wise  comparisons  on  the  color- 
word  naming  condition,  rather  than  computing  an  "interference"  score  to  control  for  the 
effects  of  TBI-related  generalized  slowing  (Perlstein  et  al.,  1998).  As  an  aUemate  means 
of  comparison,  the  proportion  of  errors  between  groups  on  this  task  showed  the  expected 
interference  eflFect,  but  the  difference  did  not  reach  statistical  significance  (Seignourel, 
Robins,  Larson,  Demery,  Cole,  &  Perlstein,  2004). 

No  significant  differences  between  controls  and  mild  TBI  participants  on  any  test 
may  have  occurred  because  the  effect  may  have  been  too  small  to  detect  with  the  study 
sample  size.  Another  reason  for  the  lack  of  significant  findings  between  controls  and  mild 
TBI  participants  may  have  been  due  to  the  significant  amount  of  time  post-injury  that  had 
elapsed  (average=20  months)  in  our  mild  TBI  group.  A  relatively  small  number  of 
individuals  remain  symptomatic  following  mild  TBI  (Alexander,  1995;  Binder,  Rohling,  & 
Larrabee,  1997).  In  fact,  some  authors  have  reported  that  most  individuals  recover 
cognitive  functioning  by  3 -months  post-injury  (Alexander,  1995;  Dikmen  &  Levin,  1993). 
Statistically  significant  differences  between  controls  and  mild  TBI  participants  may  have 
been  found  if  we  had  sampled  patients  who  were  less  than  1 -month  post-injury.  Future 
studies  should  incorporate  the  use  of  a  single  battery  of  executive  functioning  tests  at 
different  time  points  in  the  recovery  period  following  TBI  to  determine  if  and  when 
differences  in  diagnostic  accuracy  occur. 


106 
Operating  Characteristics 
Controls  and  All  TBIs 

When  considering  which  operating  characteristic  is  most  important  in  terms  of 
diagnostic  accuracy,  we  selected  positive  predictive  power  because  it  takes  into  account 
the  number  of  true  positives  among  all  test  positives  (true  positives  and  false  positives), 
whereas  test  sensitivity  measures  true  positives  among  the  patient  population  only  (true 
positives;  Retzlaff  &  Gibertini,  2000).  As  a  consultant,  the  neuropsychologist  does  not 
always  know  if  the  disorder  (TBI)  is  present  in  the  individual  they  are  testing  (N  of  one 
case).  In  fact,  in  some  cases  (e.g.,  litigating  mild  TBI),  the  referring  physician  may  not 
fiilly  know  if  the  disorder  is  present.  Basing  clinical  decision  making  on  operating 
characteristics  such  as  positive  predictive  power  mimics  real-life  clinical  decision  making 
and  should  be  considered  when  selecting  tests,  though  other  issues  are  also  important. 
Characteristics  such  as  an  understanding  of  the  measurement  error  associated  with  a  test 
should  also  be  considered.  In  some  diagnostic  situations,  erroneous  diagnostic  decision 
making  is  more  problematic  than  in  others.  For  example,  in  cases  where  eariy  detection  of 
a  neurological  disease  is  necessary  in  order  to  start  eariy  treatment  (e.g.,  cases  of  mild 
cognitive  impairment,  fast-growing  tumors),  one  should  select  tests  based  on  rates  of  error 
and  sensitivity.  In  cases  where  diagnostic  accuracy  is  less  critical  (e.g.,  assessment  of 
neuropsychological  sequelae  after  TBI),  the  implications  of  making  diagnostic  errors  are 
not  as  grave.  Given  this,  the  clinical  context  in  which  testing  occurs  should  determine 
what  test  features  receive  paramount  attention  when  selecting  neuropsychological  tests. 

We  identified  the  operating  characteristics  of  executive  functioning  tests  in  a 
sample  of  controls  and  traumatic  brain  injury  participants,  and  we  hypothesized  that 
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cognitive  psychological  tests  would  yield  higher  PPP  when  compared  to  traditional 
neuropsychological  tests  or  tests  of  everyday  executive  dysfixnction.  Partial  support  for 
our  hypothesis  was  obtained,  although  the  numerical  differences  in  PPP  between 
cognitive,  traditional  neuropsychological,  and  tests  of  everyday  executive  dysflinction 
were  not  significantly  different. 

When  controls  and  all  TBI  participants  were  pooled  together  and  performance  was 
analyzed  at  the  1 .0  SD  level  of  impairment,  the  tests  that  were  best  able  to  predict  the 
presence  of  a  TBI  were  the  computerized  WCST  (proportion  of  perseverative  errors  and 
categories  completed)  and  the  Digit  Span  backwards  test.  Results  showed  that  the 
computerized  WCST  proportion  of  perseverative  errors  variable  best  predicted  the 
presence  of  a  TBI  with  a  high  PPP  of  .95.  Based  on  these  findings,  individuals  who 
scored  at  the  cutoff  were  9.4  times  more  likely  to  have  sustained  a  TBI  than  individuals 
who  performed  below  that  level.  The  categories  completed  variable  of  the  computerized 
WCST  achieved  a  PPP  of  .92  (odds  ratio  =  5.7  times  more  likely  to  have  sustained  a  TBI), 
and  the  Digit  Span  backward  test  was  also  a  good  predictor  of  the  presence  of  TBI  (PPP 
=  .92;  odds  ratio  =  6.3). 

The  tests  that  were  poor  predictors  of  the  presence  of  a  TBI  were  the  BADS 
Modified  Six  =  Elements  (MSE)  test,  computerized  Stroop  (Color-short  incongruent),  and 
the  Digit  Span  forward  test.  Results  showed  that  the  BADS  MSE  test  was  the  poorest 
predictor  of  the  presence  of  a  TBI  with  a  PPP  of  .70  (odds  ratio  =1.2),  followed  by  the 
computerized  Stroop  -  color  short  incongruent  trial  variable  (PPP  =  .71;  odds  ratio  =  1.3) 
and  the  Digit  Span  forward  test  (PPP  =  .71;  odds  ratio  =  1 .3). 


108 

Our  findings  of  variable  rates  of  positive  predictive  power  across  executive 
functioning  measures  are  similar  to  those  of  other  researchers  who  have  studied  the 
operating  characteristics  of  executive  fijnctioning  tests  in  other  clinical  populations  (e.g., 
ADHD  children  and  adults;  Grodzinsky  &  Barkley,  1999;  Lovejoy,  Ball,  Keats,  Stutts, 
Spain,  Janda,  &  Janusz,  1999).  Grodzinsky  &  Barkley  (1999)  found  positive  predictive 
power  rates  fi-om  .65  (WCST  -  failure  to  maintain  set)  to  1.0  (Rey-Osterrieth  Complex 
Figure)  on  tests  they  administered  to  children  with  (n=64)  or  without  (n=66)  ADHD 
(prevalence  rate  =  .51).  However,  neither  the  Stroop,  Trail  Making  Test,  WCST,  nor 
Controlled  Oral  Word  Association  test  reached  PPP  rates  in  the  .90  range.  Lovejoy  et  al, 
(1999)  reported  PPP  rates  fi-om  .86  (Trail  Making  Test  part  B)  to  1.0  (Stroop,  Trail 
Making  Test  part  A,  Wechsler  Adult  Intelligence  Scale  Distractibility  Index)  on  tests  that 
were  administered  to  adults  with  (n=26)  or  without  (n=26)  ADHD  (prevalence  rate  = 
.50).  Our  clinical  population  may  be  similar  to  the  clinical  groups  used  in  the 
aforementioned  studies  given  that  brain  dysfunction  in  both  TBI  and  ADHD  may  involve 
the  fi-ontal  lobes  and  executive  systems.  These  findings  also  suggest  that  either  tests  of 
executive  dysfianction  have  different  operating  characteristics,  or  that  the  executive 
dysfunction  seen  in  TBI  vary  across  individuals  and  to  the  extent  that  any  particular  test 
measures  the  specific  executive  dysfunction  that  a  TBI  patient  has,  it  is  sensitive.  A  test's 
sensitivity  and  PPP  are  likely  related  to  the  demand  characteristics  of  the  test. 
Controls  and  Moderate/Severe  TBIs 

The  tests  that  were  best  able  to  predict  the  presence  of  a  moderate/severe  TBI 
were  the  computerized  WCST  (proportion  of  perseverative  errors),  computerized  WCST 
(Categories  Completed),  and  the  Digit  Span  test  (Backwards).  Among  all  dependent 
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variables  in  the  study,  results  showed  that  the  computerized  WCST  proportion  of 
perseverative  errors  variable  was  best  able  to  predict  the  presence  of  a  moderate/severe 
TBI  (PPP  =  93).  When  an  individual  obtained  a  score  at  the  1 .0  SD  level  of  impairment 
on  this  variable,  they  were  1 3  times  more  likely  to  have  sustained  moderate/severe  brain 
dysfiinction  than  individuals  who  performed  above  that  level.  The  computerized  WCST 
categories  completed  variable  was  second  highest  among  controls  and  moderate/severe 
TBI  participants  and  achieved  a  PPP  of  .91  (odds  ratio  =  9.2  times  more  likely  to  have 
sustained  moderate/severe  brain  dysfiinction).  The  Digit  Span  backwards  variable  reached 
a  PPP  of  .90  and  individuals  who  scored  at  the  cutoff" score  were  8.3  times  more  likely  to 
have  sustained  a  moderate/severe  TBI  than  individuals  who  performed  above  that  level. 
Overall,  the  diagnostic  accuracy  of  these  tests  appeared  good. 

Tests  that  were  poor  predictors  of  the  presence  of  a  moderate/severe  TBI  were  the 
BADS  Modified  Six  Elements  (MSE)  test,  computerized  Stroop  (Color-short 
incongruent),  and  the  Digit  Span  forward  test.  Our  findings  indicated  that  the  BADS 
MSE  test  was  the  poorest  predictor  of  the  presence  of  a  moderate/severe  TBI  with  a  PPP 
of  .50  (odds  ratio  =  .92),  followed  by  the  Computerized  Stroop  Color-short  incongruent 
trials  which  had  a  PPP  of  .62  and  odds  ratio  of  1 .5,  and  then  the  Digit  Span  Forward  test 
with  a  PPP  of  .63  (odds  ratio  =1.5). 
Controls  and  Mild  TBIs 

The  tests  that  best  predicted  the  presence  of  a  mild  TBI  were  the  computerized 
WCST  (proportion  of  perseverative  errors).  Digit  Span  test  (Backwards),  and  the 
traditional  (card)  version  of  the  WCST  (proportion  of  perseverative  errors).  Results 
showed  that  the  PPP  for  the  computerized  WCST  proportion  of  perseverative  errors 
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variable  was  best  able  to  predict  the  presence  of  a  mild  TBI  (.80).  When  an  individual 
obtained  a  score  at  the  1 .0  SD  level  of  impairment,  they  were  4.8  times  more  likely  to 
have  sustained  a  mild  TBI  than  individuals  who  performed  above  that  level.  The  Digit 
Span  Backward  test  was  second  highest  among  controls  and  mild  TBI  participants 
achieving  a  PPP  of  .75  (odds  ratio  =  3.6  times  more  likely  to  have  sustained  a  mild  TBI). 
On  the  traditional  (card)  version  of  the  WCST,  proportion  of  perseverative  errors 
achieved  a  PPP  of  .75  (odds  ratio  =  2.2  times  more  likely  to  have  sustained  a  mild  TBI). 

The  tests  that  were  poor  predictors  of  the  presence  of  a  mild  TBI  were  the 
computerized  Stroop  (Color-short  incongruent),  AX-CPT  Interference  (d'  AX  Int.  Long), 
and  the  computerized  WCST  (Categories  Completed).  Resuhs  showed  that  the 
computerized  Stroop  color-short  incongruent  trials  variable  was  the  poorest  predictor  of 
the  presence  of  a  mild  TBI  with  a  PPP  of  .44  (odds  ratio  =  .96),  followed  by  the  AX-CPT 
Interference  variable  d'  AX  Int.  Long  which  had  a  PPP  of  .44  and  odds  ratio  of  .96,  and 
then  the  computerized  WCST  categories  completed  with  a  PPP  of  .50  (odds  ratio  =  1.2). 

Our  findings  are  consistent  with  previous  research  that  evaluated  the  diagnostic 
accuracy  of  attention  measures  after  mild  TBI  (Cicerone  &  Azulay,  2002).  The  authors 
administered  a  battery  of  attention  measures  to  individuals  with  persistent  post-concussive 
syndrome  (n=32)  and  healthy  controls  (n=32;  prevalence  rate=.50)  and  found  positive 
predictive  power  rates  of  .40  (Stroop  interference)  to  .72  (Trial  Making  Test  part  A)  when 
using  a  cutoff  at  the  1 .0  SD  level  of  impairment.  In  our  controls  (n=24)  vs.  mild  TBI 
(n=20)  comparisons,  we  found  higher  positive  predictive  power  on  the  Digit  Span 
backward  (our  study/Cicerone  &  Azulay  =  .75/50)  and  Stroop  (.56/40)  tests.  We  found 
lower  positive  predictive  power  on  the  PASAT  (.55/65),  Trail  Making  Test  part  A 
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{.611.12),  and  Digit  Span  forward  test  (.46/66).  Both  studies  found  equivalent  positive 
predictive  power  on  the  Trail  Making  Test  part  B  (.67/67).  Coupled  together,  these 
studies  demonstrate  that  these  tests  have  moderately  high  rates  of  positive  predictive 
power  and  have  value  in  the  diagnostic  assessment  of  mild  TBI. 

The  differences  in  positive  predictive  power  across  computerized  versus  card 
versions  of  the  WCST  may  have  occurred  for  several  reasons.  First,  the  card  version  was 
always  administered  before  the  computer  version  in  this  study.  This  was  done  to  ensure 
that  the  concept  formation  aspect  of  the  WCST  (e.g.,  sort  by  color,  shape,  number) 
remained  novel  during  the  card  version.  The  administration  of  the  card  before  the 
computer  version  of  the  WCST  coupled  with  the  instructional  cues  that  are  provided 
before  taking  the  computerized  WCST  are  meant  to  remove  the  concept  formation  aspect 
of  the  computerized  task.  This  is  suppose  to  have  the  effect  of  placing  greater  demands 
on  the  ability  to  negotiate  the  perseverative  response  tendency.  Perseverative  errors  have 
consistently  been  the  most  sensitive  dependent  variable  on  the  WCST  (Demakis,  2003) 
and  our  computerized  version  of  the  test  places  a  strong  demand  on  perseverative 
responding.  For  example,  there  are  only  128  sorts  in  the  traditional  card  version  of  the 
WCST  while  there  are  288  sorts  in  the  computerized  version.  This  increase  in  the  number 
of  sorts  results  in  a  greater  number  of  behaviors  (e.g.,  sorts)  and  is  more  likely  to  reveal  a 
perseverative  response  tendency  in  the  participant  if  one  is  present.  The  same  can  be  said 
about  the  number  of  categories  achieved.  In  the  traditional  version,  only  six  categories 
may  be  achieved,  but  fifteen  categories  can  be  achieved  in  the  computerized  version.  This 
again  increases  the  likelihood  of  finding  an  effect  in  an  individual.  Second,  the 
introduction  of  a  5-second  time  limit  for  each  sort  on  the  computer  version  may  have  led 


112 
to  greater  perseverative  responding  and  fewer  categories  achieved  for  the  moderate/severe 
TBI  group  on  the  computer  administration  compared  to  the  card  version  of  the  WCST. 
Third,  fewer  participants  took  the  card  version  (n=37)  as  compared  to  the  computer 
version  of  the  test  (n=70)  which  may  have  resuhed  in  a  smaller  effect  size  for  the  card 
version. 

The  computerized  WCST  proportion  of  perseverative  errors  achieved  high  positive 
predictive  power  in  cases  of  mild  and  moderate/severe  TBI,  but  the  variable  categories 
completed  achieved  high  positive  predictive  power  in  moderate/severe  TBI  cases  only. 
The  finding  of  dependent  variables  with  disparate  utility  in  TBI  illustrates  the  importance 
of  considering  all  data  (dependent  variables)  fi^om  tests  before  deciding  if  a  test  is  useful  in 
select  clinical  populations.  The  findings  may  suggest  that  some  tests  may  be  administered 
in  a  manner  that  maximizes  the  acquisition  of  "relevant"  dependent  variables  and 
minimizes  time  spent  obtaining  meaningless  ones.  While  it  is  difficuh  to  see  how  the 
WCST  could  be  administered  in  its  current  form  without  some  index  of  the  number  of 
categories  that  were  completed,  tests  like  Digit  Span  may  be  altered  to  inform  the  clinician 
about  digits  backward  only.  This  cuts  administration  time  in  half  for  this  test  and  allows 
time  and  effort  for  more  meaningful  evaluative  methods.  In  the  future,  those  who  develop 
neuropsychological  tests  should  consider  which  dependent  variables  are  most  useful  and 
develop  tests  based  on  those  findings. 

The  findings  that  performance  was  worse  in  the  computerized  WCST  (where 
instructional  cues  are  given)  are  not  consistent  with  the  work  of  other  researchers  who 
provided  instructional  cues  on  the  card  version  of  the  WCST  and  subsequently  found 
improved  performance  in  schizophrenics  (Goldman,  Axelrod,  &  Tompkins,  1992).  The 
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fact  that  the  computerized  version  used  in  our  study  requires  responses  within  a  time 
deadline  may  have  been  the  reason  for  the  discrepancy.  Slowed  rate  of  information 
processing  speed  in  our  moderate/severe  TBI  group  may  have  contributed  to  an  inability 
to  consistently  make  correct  sorts. 

Discriminating  Groups 

Our  final  purpose  for  conducting  the  study  was  to  determine  if  predictor  variables 
from  neuropsychological  tests  were  able  to  reliably  discriminate  groups  (e.g.,  control, 
mild,  or  moderate/severe).  We  predicted  that  a  weighted  linear  combination  of  predictor 
variables  that  could  discriminate  groups  would  include  more  cognitive  psychological  tests 
than  traditional  neuropsychological  tests  or  tests  of  everyday  executive  dysfunction. 

The  findings  showed  that  only  the  computerized  WCST  proportion  of 
perseverative  errors  variable  was  usefial  in  discriminating  groups.  A  perseverative 
response  tendency  is  the  repetition  of  a  behavior  even  when  it  is  no  longer  appropriate 
(Kimberg,  D'Esposito,  &  Farah,  1997b).  More  specifically,  a  stuck-in-set  perseveration  is 
the  inappropriate  maintenance  of  a  current  category  or  framework  (Sandson  &  Albert, 
1984),  Stuck-in-set  perseverations  may  be  considered  a  defect  in  response  inhibition 
(Roberts  &  Pennington,  1996)  and  are  indicative  of  executive  dysfunction.    However, 
Kimberg,  D'Esposito,  &  Farah  (1997b)  suggest  that  a  defect  in  working  memory  may 
account  for  perseveration.  Cohen  &  Servan-Schreiber's  (1992)  asserted  that  working 
memory  deficits  are  impairments  in  the  ability  to  construct  and  maintain  internal 
representations  of  information  (context  maintenance)  that  are  to  be  held  in  mind  in  such  a 
form  that  they  can  be  used  to  mediate  appropriate  behavioral  responses.  Based  on  this 
idea,  Kimberg  et  al.  (1997)  proposed  a  spread  of  activation  model  of  working  memory 
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that  attributed  perseverative  responding  not  to  inhibitory  dyscontrol,  but  to  an  inability  to 
utilize  context  (criterion  change)  to  actively  select  a  new  response  that  fits  the  new 
criterion.  In  our  study,  moderate/severe  TBI  patients  performed  very  poorly  on  measures 
of  working  memory  (N-Back  High),  suggestive  that  they  may  have  made  more 
perseverative  response  due  to  a  defect  in  context  maintenance.  In  addition,  an  analysis  of 
the  structure  coefficients  in  the  discriminant  fianction  analysis  showed  a  negative  (small) 
correlation  between  the  discriminant  function  and  performance  on  the  Digit  Span 
backward  test  -  a  measure  of  working  memory.  Further  indicating  that  faulty  working 
memory  may  play  an  important  part  in  the  behavior  of  perseveration.  Future  studies 
should  attempt  to  flirther  distinguish  the  role  of  working  memory  in  perseveration  and 
directly  address  the  concept  of  perseveration  in  light  of  these  findings. 

One  of  the  greatest  challenges  facing  executive  functioning  test  developers  is  the 
task-  impurity  problem.  This  refers  to  the  notion  that  executive  functioning  tests  also 
make  demands  on  a  variety  of  other  cognitive  skills  or  fianctions  that  are  supported  by 
brain  structures  that  are  quite  independent  of  frontal  cortex  (Rabbitt,  1997).  The 
development  of  cognitive  psychological  tasks,  such  as  those  used  in  the  present  study,  is 
an  attempt  to  create  experimental  paradigms  that  elucidate  the  nature  of  defective 
functioning  of  the  component  processes  of  executive  functioning  (e.g.,  inhibitory  control, 
working  memory,  etc.)  after  TBI. 

While  cognitive  psychological  tests  may  provide  "purer"  information  about 
executive  functioning/control,  it  is  unlikely  that  dependent  variables  with  greater 
conceptual  "purity"  provide  greater  sensitivity  or  predictive  power  than  omnibus  tests  that 
make  demands  on  multiple  functional  systems.  A  test  that  has  multiple  demand 
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characteristics  requires  the  integrity  of  an  entire  functional  system,  and  if  any  element  of 
that  system  is  impaired,  it  is  likely  to  manifest  as  impaired  system  (test)  performance.  The 
dilemma  that  clinicians  face  is  that  omnibus  tests  appear  to  be  more  sensitive  to  the 
presence  of  brain  dysfunction  and  are  subsequently  better  at  detecting  the  presence  of  a 
disorder  when  it  is  present;  however,  omnibus  tests  do  little  in  terms  of  explaining  the 
unique  variance  that  exists  between  executive  flinctioning/control  measures.  Purer  tests 
more  clearly  inform  the  clinician  about  the  integrity  of  component  processes  of  executive 
behavior,  but  they  unfortunately  are  not  likely  to  provide  the  sensitivity  and  predictive 
power  of  omnibus  tests.  This  result  was  observed  in  the  present  study  -  the  omnibus  test 
WCST  yielded  the  greatest  PPP  while  an  arguably  pure  test  of  working  memory,  such  as 
the  N-Back,  had  high  PPP  but  not  as  high  as  the  WCST.  These  findings  support  the  idea 
that  clinicians  should  select  tests  based  on  the  nature  of  the  referral  question  (e.g..  Is  there 
a  disease  present?  vs.  Does  this  person  have  executive  dysfunction?). 

According  to  Sbordone  (1996),  ecological  validity  refers  to  "the  functional  and 
predictive  relationship  between  the  patient's  performance  on  a  set  of  neuropsychological 
tests  and  the  patient's  behavior  in  a  variety  of  real- world  settings  (e.g.,  home,  work, 
school,  community)"  (p.  16).  While  neuropsychological  tests  have  been  generally  useful  in 
detecting  and/or  quantifying  brain  dysfunction,  the  tests  continue  to  lack  relevance  in 
predicting  real-world  behavior  (Wilson,  Evans,  Alderman,  Burgess,  &  Emslie,  1996).  We 
included  a  broad  range  of  executive  functioning/control  tests  that  we  considered 
differentially  ecologically  valid.  For  example,  we  included  tests  that  were  thought  to  have 
low  ecological  validity  but  high  sensitivity  (e.g.,  cognitive  psychological  tests).  We 
selected  tests  touted  as  measures  of  high  ecological  validity  (Wilson  et  al.,  1996)  but  with 
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predicted  low  sensitivity  (e.g..  Tests  of  everyday  executive  dysfunction).  We  felt  that 
traditional  neuropsychological  tests  had  generally  low  ecological  validity,  albeit  higher 
than  cognitive  psychological  tests,  and  adequate  sensitivity,  albeit  lower  than  cognitive 
psychological  tests.  Results  showed  that  while  tests  of  everyday  executive  dysfunction  are 
touted  as  having  good  ecological  validity,  they  lacked  sensitivity  and  predictive  power  in 
the  present  study  to  warrant  use  in  outpatient  clinical  settings  with  individuals  with  chronic 
TBI.  These  findings  notwithstanding,  tests  of  everyday  executive  dysfunction  may  yield 
higher  rates  of  sensitivity  and  predictive  power  in  individuals  with  acute  TBI.  Test 
developers  should  continue  attempting  to  expand  the  use  of  neuropsychological  tests  in 
predicting  real-world  behavior. 

While  the  task  impurity  problem  and  issues  of  ecological  validity  are  important 
obstacles  in  the  assessment  of  executive  functioning/control,  probably  the  most 
controversial  yet  important  issue  is  the  problem  of  the  definition  and  use  of  the  term 
executive  fiinctioning.  Although  Lezak  (1995)  defined  executive  functioning  as  "the 
capacity  that  enables  a  person  to  engage  successfully  in  independent,  purposive,  self- 
serving  behavior"  (p.  42),  this  definition  does  little  in  terms  of  clearly  elucidating  the 
component  processes  subsumed  under  executive  functioning/control.  There  is  little 
question  that  the  term  'executive  functioning'  facilitates  discussions  about  a  nondescript 
set  of  abilities  that  sit  atop  the  cognitive  hierarchy.  However,  the  term  does  little  in  terms 
in  promoting  a  more  precise  understanding  of  the  component  processes  that  comprise 
executive  behavior.  Executive  functioning/control  is  the  final  common  pathway  for  a 
multitude  of  subordinate  behaviors  that  must  be  intact  in  order  for  the  organism  to 
perform  executive  behavior.  Neuropsychologists  must  be  able  to  understand  where  in  the 
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cognitive  hierarchy  impaired  functioning  is  occurring  (e.g.,  perceptual  systems,  updating, 
active  maintenance,  etc.)  in  order  to  best  understand  the  nature  of  executive  dysfunction. 

Limitations  of  this  study  should  be  considered.  The  sample  size  used  in  this  study 
was  small.  In  light  of  past  research  that  indicates  a  small  but  consistent  effect  in 
performance  on  neuropsychological  tests  between  controls  and  mild  TBI  participants 
(Binder  et  al.,  1997),  future  studies  should  include  larger  samples  to  determine  if 
significant  differences  between  controls  and  mild  TBI  participants  exist  on  the  tests  that 
were  unique  to  this  study. 

Although  the  retrospective  method  of  collecting  information  about  brain  injury 
severity  has  been  validated  in  past  studies  (King  et  al.,  1997;  McMillian,  Jongen  &. 
Greenwood,  1996),  having  acute  medical  records  on  all  TBI  participants  is  an  advantage 
when  classifying  participants  in  research  studies.  A  limitation  of  the  current  study  is  that 
only  7  of  42  participants  who  were  treated  in  the  ER  immediately  following  their  injury 
brought  medical  records  at  the  time  of  study  participation. 

The  inclusion  of  acute  injury  data  such  as  Glasgow  Coma  Scale  score,  duration  of 
loss  of  consciousness  or  post-traumatic  amnesia,  or  positive  cranial  nerve  III  sign  (dilated 
pupil),  would  be  very  important  in  improving  diagnostic  accuracy  and  in  discriminating 
between  groups. 


Appendix 
Structured  Clinical  Interview  Form 


Date: 


Subj.  Name: Subj.  #: Control  or 

TBI  Gender:  M  or  F*Race*: DO  B.: (Age ) 

Handedness:  L  R  Mixed 

Date  of  initial  contact 
Date  of  2"'' contact 
Date  of  3"*  contact: 
Date  of  4*  contact: 

Does  the  patient  meet  inclusion  criteria  for  behavioral  study?  Yes  or  No 

Has  the  patient  signed  the  informed  consent?  Yes  or  No 

Date  signed  informed  consent  

Has  the  patient  been  scheduled  for  behavioral  study?  Yes  or  No 

Date  patient  to  participate  in  behavioral  study  

Date  patient  participated  in  the  behavioral  study  

Does  the  patient  meet  inclusion  criteria  for  the  fMRI  study?  Yes  or  No 

Has  the  patient  signed  the  informed  consent?  Yes  or  No 

Date  signed  informed  consent  

Has  the  patient  been  scheduled  for  fMRI  study?  Yes  or  No 

Date  patient  to  participate  in  fMRI  study  

Date  patient  participated  in  fMRI  study  

Has  the  patient  been  sent  his/her  payment  for  participation?  Yes  or  No 

May  the  patient  be  contacted  for  future  research  studies?  Yes  or  No 
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Date: 


Subj.  Name: 

TBI  Gender:  M  or  F* Race*: 

Subj.  #:                   Control  or 
DOB.:                  (Age          ) 

Handedness:  L  R  Mixed 
Address: 

Home  Ph.#  (              ) 

Work 

Ph.#(              ) 

Years  of  education  (YOE)  Pt*: 

Y.O.E.  Mother:              Y.O.E.  Father: 

Pt's  Occupation  (Be 
specific)*: 

Mother's 

Occupation  (Be  specific): 

Father's  Occupation  (Be 
specific): 

Pt's  level  of  fijnctioning  before  TBI**: 
Pt's  level  of  functioning  after  TBI**:_ 


Pt's  Marital  Status:  S  M  D  W Residence*:  <  or  >  2500         Legal  status*^ 
Did  the  pt.  go  to  the  ER  as  a  result  of  accident? 
Was  Pt.  admitted  to  the  hospital  from  the  ER? 

If  yes,  duration  of  hospitalization  (Days): 

Mechanism  of  injury*  * : Type  of  injury*  * : 

Time  since  injury  (Months): 


Yes  or  No 
Yes  or  No 


LOC  associated  with  accident? 

If  Yes,  est.  duration  of  LOC  (in  minutes): 
PTA  associated  with  accident? 

If  Yes,  est.  duration  of  PTA  (in  minutes): 


Yes  or  No 


Yes  or  No 


Estimated  duration  of  retrograde  amnesia  (in  minutes): 


Estimated  duration  of  anterograde  amnesia  (in  minutes): 
Prior  Medical  History  (Before  TBI)? 


Yes  or  No 
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Prior  History  of  Neurological  insult  (e.g.  TBI,  CVA)?  Yes  or  No 


Prior  History  of  serious  Psychiatric/Psychological  Disorder?                        Yes  or  No 
If  yes,  Dx: ^When: 


How  treated: How  long: 

If  female,  are  you  currently  pregnant?  Yes  or  No 
If  No,  is  there  a  possibility  that  you  are?  Yes  or  No 
Current  medications: __^ 

Medication  or  recreational  drugs  taken  within  the  last  24  hours: 


Overall  GPA  in  High  School: In  College: 

Problem  Areas?  Prior  Hx.  of  LD? 


Hx.  of  Special  Education  Classes? 
Hx.  of  ADD  or  ADHD: 


Hx.  of  drug  or  alcohol  abuse: 

Average  number  of  alcoholic  drinks  per  week: 

Have  you  participated  in  research  studies  before?  Yes  or  No 

If  yes,  specify__ 

When  was  the  last  time  you  had  neuropsychological  testing? 

Do  you  have  any  metal  clips  or  plates  in  your  body  or  past  bridgework?        Yes  or  No 
What  are  the  best  days  and  times  for  you  to  participate  in  the  research  study? 


Will  you  be  willing  to  sign  a  release  for  medical  records  if  necessary?  Yes  or  No 

If  yes,  date  signed 
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Level  of  functioning  before  TBI 

0=Pt.  employed  or  student 

l=Pt.  '/2  time  employed  or  Vi  time  student 

2=Pt.  not  employed  or  attending  school 

Level  of  functioning  after  TBI 

0=Pt.  returned  to  premorbid  activities  with  no  modification 
l=Pt.  returned  to  premorbid  activities  with  moderate  modifications 
2=Pt.  unable  to  return  to  premorbid  activities 
Litigation  status 

O=not  in  litigation,  no  future  plans  of  litigation 
l=not  in  litigation,  future  plans  of  litigation 
2=in  litigation 
3=past  litigation  related  to  accident 

Mechanism  of  injury 

0=Motor  vehicle  accident 
1=MVA  vs.  pedestrian 
2=Fall 
3=Assauh 

4=Sport/recreation  injury 
5=0ther,  specify 

Type  of  injury 

0=Acceleration 

l=Deceleration 

2=Pure  acceleration  with  blunt  head  trauma 

3=Pure  deceleration  with  blunt  head  trauma 

4= Acceleration  and  Deceleration  without  blunt  head  trauma 

5=Acceleration  and  Deceleration  with  blunt  head  trauma 
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